Introduction
Let's face it, using language models (LLMs) to classify products or search queries has become a routine task. However, these models often struggle to adapt to the predefined vocabulary of brands, colors, and categories that a system allows. Consider the Wayfair WANDS e-commerce dataset: how do you classify a query like "wood coffee table" among hundreds of possible categories?
The Problem with Traditional Classifications
Traditional methods rely on structured outputs. You define a list of allowed values and ensure your models adhere to it. In the Wayfair example, this means creating a massive set of legal values—a process that is both time-consuming and resource-intensive. Even with tools like Pydantic in Python, the effort remains substantial.
Take this example with Pydantic:
```python from typing import Literal from pydantic import BaseModel, Field
FullyQualifiedClassifications = Literal[ 'Furniture / Bedroom Furniture / Beds & Headboards / Beds', ... # 500 others ]
class QueryClassification(BaseModel): classifications: list[FullyQualifiedClassifications] = Field(description="Possible classification for the product.") ```
An Innovative Alternative: Hallucination
Instead of bogging down in complex models, why not ask a "dumb" LLM to create plausible fictitious classifications? This is the idea behind controlled hallucination. By generating fictional and original classifications, we circumvent the limitations of classic models while reducing costs.
Hallucination Example
``python hallucination_prompt = f""" Your task is to create novel, never seen before classifications for a search query. Here's the query: wood coffee table """ ``
The idea is simple and effective: ask a language model to design classifications that have never been seen before, tailored to the given context.
Benefits and Limitations
Benefits
- Resource Savings: Hallucination allows the use of smaller, less costly models.
- Flexibility: No need to maintain a massive list of predefined classifications.
- Innovation: This approach leverages the often-underutilized creativity of the model.
Limitations
- Reduced Control: Hallucination can lead to unexpected classifications.
- Need for Validation: Results must be verified to ensure relevance and compliance.
Use Cases and Economic Impact
Consider an e-commerce company that needs to classify thousands of new products daily. By integrating controlled hallucination, it could reduce its classification costs by 30 to 50%, while increasing processing speed.
Conclusion
Controlled hallucination is a game-changer in data classification. It offers a viable, cost-effective alternative to traditional methods while providing unprecedented flexibility. Curious to see how this can transform your project?
Let's discuss your project in 15 minutes.