If it was me, I would drive it from LGA, since you are driving away from downtown, on Thursday. Should not be an all day deal. However, the return probably might be problematic, heading into the city. I would fly out Monday, after spending night near airport. But that's me, and I enjoy driving in new areas, as long as it's moving.
Obviously, like you're thinking about doing, flying to White Plains is viable. Looks like about $350-400 round trip back to LGA. That's a lot to pay to travel 30 miles, then get a rental to drive.
You might want to consider an Uber from LGA to a rental car place some distance away from the city, let them deal with the city traffic for so many miles, then drive in less congested area.
And another viable option is helicopter flight. They can be rented/private charter for around $250 for 30 minutes minimum, West Point area helipad/small airport should be under 30 minutes and closer than White Plains. Same for private charter plane. Then rent a car or Uber.
