Cultural references can be picked up without reference to the original work--at least sufficiently for the purpos. .I "get" cultural references to an awful lot of books and tv shows and movies (and video games, and comic books, and ...) that I've never consumed directly. No doubt there are nuance I miss, but people who consumed the materials directly may skim over, forget, or not understand aspects that become cultural references. There are classic works of fiction where I honestly don't remember whether I read them or not because I know them well enough to understand their cultural importance, but if I did, it was long enough ago that the details have faded. To some extent, that kind of cultural reference is something like herd immunity: if enough people have direct experience, then you can pick up the benefits indirectly.