According to Gartner analyst Dan Ayoub, 'Readily available, high-quality GenAI applications are now capable of creating photo-realistic video content that can deceive or mislead an audience. Given the low barriers to entry in using these increasingly sophisticated tools, developing a methodological approach to detecting GenAI deepfake content has become necessary.
They are being advertised alongside new offerings, such as DarkGemini and TorGPT, now equipped with multimodal capabilities, including image-generation services.However, the report notes that many of the ChatGPT-lookalike services offered on the cybercrime underground are little more than 'jailbreak-as-a-service' front-ends designed to trick commercial LLMs into providing unfiltered responses to malicious queries.