Beneficial effects of omega-3 fatty acid intake on cognition are under debate as some studies show beneficial effects while others show no effects of omega-3 supplementation. These inconsistencies may be a result of inter-individual response variations, potentially caused by gene and diet interactions. SorLA is a multifunctional receptor involved in ligand trafficking including lipoprotein lipase and amyloid precursor protein. Decreased SorLA levels have been correlated to Alzheimer's disease, and omega-3 fatty acid supplementation is known to increase SorLA expression in neuronal cell lines and mouse models. We therefore addressed potential correlations between Sorl1 and dietary omega-3 in SorLA deficient mice (Sorl1^−/−) and controls exposed to diets supplemented with or deprived of omega-3 during their entire development and lifespan (lifelong) or solely from the time of weaning (post weaning). Observed diet-induced effects were only evident when exposed to lifelong omega-3 supplementation or deprivation as opposed to post weaning exposure only. Lifelong exposure to omega-3 supplementation resulted in impaired spatial learning in Sorl1^−/− mice. The vitamin C antioxidant capacity in the brains of Sorl1^−/− mice was reduced, but reduced glutathione and vitamin E levels were increased, leaving the overall antioxidant capacity of the brain inconclusive. No gross morphological differences of hippocampal neurons were found to account for the altered behavior. We found a significant adverse effect in cognitive performance by combining SorLA deficiency with lifelong exposure to omega-3. Our results stress the need for investigations of the underlying molecular mechanisms to clarify the precise circumstances under which omega-3 supplementation may be beneficial.