cross some some strange model behaviour.

In one of the applications that I work on we have title generation for the chats, just like all AI apps that work with chats. And it’s using a fairly basic prompt, which does something like this (some parts are removed):

<role>
Chat title generator
</role>

{language_instruction}

<requirements>
• Maximum 6 words
• Capture main topic/question
• Descriptive but concise
• NO quotes or special characters
• Use query's language
</requirements>

<output_format>
Return ONLY the title
NO other text
</output_format>

This is nothing special, nothing strange. But the model tries to generate some SQL queries and/or function names. Which is something I have never seen before.

The most crazy part of is it, that the user query wasn’t that crazy:

I just asked about Mr Smith, which is part of my demo dataset locally. Really strange, if I find more of this behaviour I’ll write about it. That’s it for now.