Importance Being physically active is protective against dementia. Yet, it is unknown when during the adult life course physical activity is most associated with dementia risk. Objective To determine whether higher physical activity levels in early adult life, midlife, or late life are associated with lower risk of all-cause or Alzheimer disease (AD) dementia. Design, Setting, and Participants This prospective cohort study used data from the Framingham Heart Study Offspring cohort. The offspring of participants in the original Framingham Heart Study cohort who were dementia free and had physical activity measured at baseline (early adult life [1979-1983], midlife [1987-1991], or late life [1998-2001]) were followed up for a mean (SD) of 37.2 (7.1), 25.9 (8.5), or 14.5 (6.6) years for the development of incident all-cause or AD dementia until December 31, 2023. Exposure Physical activity was self-reported using the physical activity index, a composite score weighted by hours spent sleeping and in sedentary, slight, moderate, or heavy activities. Physical activity was divided into quintiles (Q). Main Outcome and Measure All-cause and AD dementia were classified by expert consensus based on established diagnostic criteria. Results This study included 1526 early adult–life (mean [SD] age, 36.7 [4.7] years; 821 [53.8%] female), 1943 midlife (mean [SD] age, 54.0 [5.8] years; 1010 [52.0%] female), and 885 late-life (mean [SD] age, 71.0 [4.5] years; 473 [53.4%] female) participants. There were 567 cases of incident all-cause dementia during follow-up. Higher levels of midlife and late-life physical activity were associated with lower risk of all-cause dementia. Midlife and late-life physical activity levels in Q4 or Q5 were associated with lower risk of all-cause dementia compared with Q1 (midlife Q4: HR, 0.60; 95% CI, 0.41-0.89; midlife Q5: HR, 0.59; 95% CI, 0.40-0.88; late-life Q4: HR, 0.64; 95% CI, 0.42-1.00; late-life Q5: HR, 0.55; 95% CI, 0.35-0.87). There were no associations between early adult–life physical activity and dementia risk. Findings were similar for incident AD (369 cases). Conclusions and Relevance In this cohort study of adults in the Framingham Heart Study Offspring cohort, higher levels of midlife and late-life physical activity were associated with similar reductions in risk of all-cause and AD dementia. These findings may inform future efforts to delay or prevent dementia through timing interventions during the most relevant stages of the adult life course.