Ok, let me put this out there from the perspective of the youngest of four boys. My mom was 38 when I was born and my dad was 40. My oldest brother is 15 yrs older than me and my closest brother is 4 yrs older than me. The number one thing I remember when I was really young is that everybody else's parents were in their 20's or so, and my parents looked ancient beside them when ever they came to school functions. More than once I had to endure voices around me snickering when they would see the blatent age difference in my parents and theirs... my friends would meet them and say "Damn dude, is that really yer dad? He looks like he is as old as my grandpa."
Another thing that sticks out is how my grandparents died before I was 20. I fee like I really missed out on a lot of quality time with them, as well. As I got older, my parents health became an issue. My dad had smoked all his life so by the time I had kids all they remember of their grandfather is that he was this ancient guy who dragged around an oxygen bottle everywhere he went. He passed when I was 35. I still miss him to this day and wish we had been able to spend more time together.
Now, My mom far outlived Pops, and lived to the ripe old age of 93. So, another pressure of being the youngest, is that when you have a parent that lives to be in their 90's is that everybody looks to you, since you are the last kid still making a living, and the others are either retired on fixed incomes or about to be. More than once, after mom had to live in an old folks facility, the financial decisions would fall to me. And more than once I was the only one capable of handling the money side of things since my siblings live on tight budgets. Hell my older brother still calls me up occasionally to help him out financially since I am the only one still working.
Overall, you do you, and what is best for you and yours, but be aware that there is more than one way to look at this situation you may be getting into... Either decision you make, I am sure it be for the best.