Chan was born on 7 April 1954 in Hong Kong to Charles and Lee-Lee Chan, refugees from the Chinese Civil War. His parents were so poor they had to borrow money from friends to pay the doctor. His parents nicknamed him Pao-pao, because the energetic child was always rolling around. His parents worked for the French ambassador in Hong Kong, and Chan spent his formative years within the grounds of the consul’s residence in the Victoria Peak district.
Chan attended the Nah-Hwa Primary School on Hong Kong Island, where he failed his first year, after which his parents withdrew him from the school. In 1960, his father emigrated to Canberra, Australia to work as the head cook for the American embassy,
and Chan was sent to the China Drama Academy, a Peking Opera School run by Master Yu Jim-yuen. Chan trained rigorously for the next decade, excelling in martial arts and acrobatics. He eventually became part of the Seven Little Fortunes, a performance group made up of the school’s best students, gaining the stage name Yuen Lo (元樓) in homage to his master. Chan became close friends with fellow group members Sammo Hung and Yuen Biao, and the three of them later became known as the Three Brothers or Three Dragons. After entering the film industry, Chan along with Sammo Hung got the opportunity to train in hapkido under the grand master Jin Pal Kim, and Chan eventually attained a black belt.