In the current study we aimed to determine which cognitive skills play a role when learning to program. We examined five cognitive skills (pattern recognition, algebra, logical reasoning, grammar learning and vocabulary learning) as predictors of course-related programming performance and their generalised programming performance in 282 students in an undergraduate introductory programming course. Initial skills in algebra, logical reasoning, and vocabulary learning predicted performance for generalised programming skill, while only logical reasoning skills predicted course-related programming performance. Structural equation modelling showed support for a model where the cognitive skills were grouped into a language factor and an algorithmic/mathematics factor. Of these two factors, only the algorithmic/mathematics factor was found to predict generalised and course-related programming skills. Our results suggested that algorithmic/mathematical skills are most relevant when predicting generalised programming success, but also showed a role for memory-related language skills.