This study aimed to investigate the impact of cancer from family caregivers’ perspective, based on a comparative analysis of caregiving burden and distress, among four types of caregivers. The sample included caregivers of persons with cancer, dementia, diabetes, or frail elderly from a nationally representative sample of 606 caregivers. Although the four different types of caregivers were comparable in most sociodemographic characteristics, caregiving involvement and caregiving outcomes differed among the caregiving groups. Both cancer and dementia caregivers reported greater levels of physical burden and psychological distress than other caregivers, after controlling for sociodemographic and caregiving involvement (i.e., level of burden and caregiving duration) factors. The comparative analysis provided a systematic review of cancer caregiving in the context of three other types of caregiving. Given the high levels of burden and distress, greater emphasis should be placed on developing social service policy and practice for cancer caregivers.