Yup, I would recommend using CALayers for the whole UI, pretty much. Back when Core Animation first came out I wrote a framework for building board/card games:

It hasn’t been updated since 2011 so it probably needs some fixes for API changes, but I still think the basic design is pretty well thought out. The goal was that you could implement a simple game UI in <200 lines of code, and there are examples included for checkers, tic-tac-toe, Klondike solitaire, and a few others.

—Jens