The Limits of LLM-Generated Unit Tests

Developers often ask LLMs like OpenAI Codex to write tests for their code - and they do. The tests compile, run, and even pass, giving a sense of confidence. But how much can we really trust those tests? If an LLM only sees the source code and a few comments, does it truly understand what the software is supposed to do?