it has already been shown how they make mistakes and provide dangerous information.

To leave that unsupervised and unchecked is irresponsible.

But the problem becomes who does the checking. AI developers won't find the same flaws as a person with no AI financial interest.