High Speed Railway (HSR) has been suggested as one sustainable alternative for acquiring regional and nationwide mobility. In order to maintain the sustainability of HSR also within changing socioeconomic structures, it is important to secure its long-term demand. Therefore this study considered the diffusion theory which explains technology adoption in the marketing area, as a basis for explaining HSR ridership increasing by year. Our target area for study is Taiwan High Speed Rail (THSR). By using the monthly ridership data of THSR between 2007 and 2015, we calculated yearly Adoption Effects which describes the spread of THSR among new users. We consider this effect to be a key element to secure long-term demand. Moreover this study demonstrated that city heterogeneity such as social, economic and geographical characteristics influence the Adoption Effect. Based on our findings, we conclude that specifically improving accessibility should be considered as a policy measure to help stabilizing long-term demand especially in an aging society.