DescentBB

Does anyone know where I can find 3D maps of all of the Descent 2 levels?

Just to clarify, are you talking about something like this?
https://www.kixor.net/games/descent_levels/
That one is only for D1 and doesn't include in-built levels... I'm not aware of a comparable tool for D2 but it wouldn't be much more difficult to build one.

That would be neat but I would also like to see important parts labeled, like the keys, key doors, reactor room, exit tunnel, etc.

Not sure if anyone bothered to go THAT in depth as far as making a guide is concerned. You could get pretty much that if you opened the levels in DLE and explored them visually though

Try finding the Descent 2 guide that came out in the 90's.