That sounds absolutely miserable. And unfortunately not uncommon at all. Yes, "our" interests were indeed just his in the end, but he managed to lure me in by latching onto mine in the beginning. In fact "we" followed a teacher named Guy Finley, who between his own talks and student talks had an activity on Sun, Mon, Weds, Fri, and Sat. If anything was going on at the same time as Guy Finley, we had to miss it because our personal growth was (obviously) more important than family events or anything else.
Thank you for sharing your story. It's amazing to me how many of us have been through this. If not for the internet, I don't think we'd know how wide-spread it truly is.