We do not have direct experience of physical things: evidence is theory-laden. That is well-understood and generally regarded as true. Much less appreciated is that we do not have direct experience of abstract things either: self-evidence is theory-laden too.
The empiricist intuition is that as we approach the things of which we have “direct experience,” our beliefs become more certain, obvious, and less prone to error. It is ironic, then, that at the very end of this chain are qualia, perhaps the only things we could be said to “experience directly,” and they are among the least understood phenomena of all.