Civic EG

The Haynes doesn't really help. It just says to pry the cover off below the housing, which I did, and then twist out the bulb and pull... but I can't get my fingers anywhere near the thing.

That's how you get it out. You have to pry it out and pull hard to the point where it feels like you're about to break stuff. There's no other way, even if you take off the entire spoiler.