James Casey came to Canada as part of the Talbot Emigrant Scheme in 1831. He carried with him a letter of introduction from Colonel Talbot's mother and their landlord, Reginald Talbot. According to author, June Callwood in her book, "Portrait of Canada", "(Colonel) Thomas Talbot screened applicants according to his prejudices. Highlanders, Quakers, half-pay officers, Americans and most Irish got the most remote and least arable land. He transacted business after noon, reserving the rest of the day drinking." James became so disheartened by the loneliness and desolation of settler life during this period that he wrote a letter to his Irish relatives for money to return to Ireland. He gave the letter to his wife for post; however, she did not do so and the family remained in Canada. James' brother, William Casey, arrived in 1837 or early 1838 from Ireland and life became more tolerable.