It's probably been 10 years ago, but Pop's would take my brothers and I every October. Occasionally had fellow fishing groups. We'd park/dock out of Ocracoke and stay on Portsmouth. October is flounder season and we would kill it. Threw a gill net across a small inlet (probably illegal then and definitely now) and would flounder gig over night. Would bring back a bunch of flounder every year, not including food expenses cause we ate fresh cooked flounder round the clock. (12" use to be keepers) Line fish with cast net caught menhaden during the day, and hook up with the monsters. Best times ever. We even kept steel shed frame posts over the berm we used every year to steady our tent. Wind is helacious out there. One year woke up with a foot of water over whole campsite. Noreaster blew in and covered us up.
Visited a church on Portsmouth once, surreal experience. Empty church with open door for visitors. Made me feel real good about our founders, that a church was an essential to being established. I believe I saw the lighthouse that got fantastically moved, could be wrong.
I'd encourage you to take the trip, especially if you are taking your kids. Memories with my family are beautiful, I was younger, but so much appreciate my Dad for doing it.