I lived in SD for 6 years while I was in the Navy (just discharged 2 years ago, so it's not like I was there 30 years ago!). Personally, I love SD!
Here are the places I would rec. hitting up:
1) USS Midway (if you like military stuff) -- it's an old retired AC that has a pretty decent walking tour. It's a little pricey so if that's a concern you might not feel like you got your money's worth, though.
2) Old Town. It's not nearly as good as it was 5 years ago or so but it's still pretty kewl. Parking can be fairly rough at times but if you park a few blocks away and just walk it's a nice walk. There are a couple good Mexican joints in that area. The Old Mormon Battalion Fort is there too, as is an officially recognized haunted house. I don't recall the name, but just ask around you won't have problems finding it. They do guided tours several nights of the week.
3) Skip the zoo... it's way too huge and tiring and IMHO it's sorta crummy (I know it's Rep, I just don't really agree) -- go to the Wild Animal Park instead. They have a really kewl tram thing that circles the park that provides some really kewl sights.
4) Yeah, you gotta go see Shamu -- all kids like Shamu. I doubt the younger ones will even notice the beer stuff.
5) Go down to Mission Bay, rent some bikes from one of the local shops there and take a nice relaxing ride around the Bay. That's a fun one we've done several times. You can also ride the roller coaster right there on the beachfront and then go strolling down the boardwalk. You should take note, though, of all the places I've mentioned here, Mission Bay is definitely the most dangerous area. Try not to stay there into the evening. During the day as long as you're alert you shouldn't have any problems. Also DO NOT GO THERE ON SUNDAY. One of the parks in the area is a known hang-out for drug dealers and thugs... only on Sundays, though... I have no idea the details as to the why but the Navy forbade us from being in that park on Sundays due to the problems they've had in the past. I can't tell you which park it is... I've been there once but I forget what it's called, so it's probably just best to avoid MB on Sundays altogether.
6) If your family likes Whale Watching, there are some fun ocean-going tours you can get in on for pretty cheap. Honestly, though, don't expect to see much of whales (a little maybe, at a distance) but just the ride at sea and seeing all the dolphins is pretty kewl.
7) You can go on an evening tour around the San Diego Bay on a boat. It includes dinner. I've never done it, but my impression it's that it's a little up-scale and a bit pricey. Not one for kids.
8) If you want to spend a little eating good seafood, I like "The Blue Crab" in "Point Loma" (just north of Downtown). It's also on the waterfront, although it is in a bit of a shabby alley-looking location. The inside is nice though.
Also, just so you know... you can eat ANYWHERE in SD without having to worry about your dress. They are very casual diners in SD. No worries.
If I think of something else I add it.