Building AI-Powered Semantic Search in Laravel with OpenAI Embeddings, Scout, and Pinecone

Why semantic search matters for modern Laravel apps

Imagine a user typing “how to reset a forgotten password in Laravel 10” and instantly receiving the exact documentation snippet, related forum posts, and even a code example that matches the intent—not just the keyword “reset”. Traditional LIKE queries struggle with this nuance, but a vector‑based semantic engine can rank results by meaning. According to a 2023 AI adoption report, 68% of developers consider AI‑enhanced search a top priority for improving user experience.

Understanding OpenAI embeddings

OpenAI’s embedding models transform any text into a dense vector of 1,536 floating‑point numbers (for text‑embedding‑ada‑002). These vectors capture semantic relationships: “reset password” and “forgotten credentials” end up close in Euclidean space. Laravel developers can call the API directly from a service class, store the resulting vectors, and later query them for similarity.

use GuzzleHttp\Client;\n\nclass OpenAIEmbeddingService\n{\n    protected $client;\n\n    public function __construct()\n    {\n        $this->client = new Client([\n            'base_uri' => 'https://api.openai.com/v1/',\n            'headers' => [\n                'Authorization' => 'Bearer '.env('OPENAI_API_KEY'),\n                'Content-Type' => 'application/json',\n            ],\n        ]);\n    }\n\n    public function embed(string $text): array\n    {\n        $response = $this->client->post('embeddings', [\n            'json' => [\n                'model' => 'text-embedding-ada-002',\n                'input' => $text,\n            ],\n        ]);\n        $data = json_decode($response->getBody(), true);\n        return $data['data'][0]['embedding'];\n    }\n}\n

The service returns a plain PHP array that can be JSON‑encoded before sending to Pinecone.

Setting up Laravel Scout for vector indexing

Laravel Scout provides a clean, driver‑based interface for full‑text and now vector search. After installing Scout, you replace the default driver with a custom Pinecone engine.

composer require laravel/scout\nphp artisan scout:install\n

Next, publish the Scout config and add a new driver entry:

// config/scout.php\n'driver' => env('SCOUT_DRIVER', 'pinecone'),\n\n'pinecone' => [\n    'api_key' => env('PINECONE_API_KEY'),\n    'environment' => env('PINECONE_ENVIRONMENT'),\n    'index' => env('PINECONE_INDEX'),\n],\n

Now create a model that implements Searchable and defines the vector field.

use Laravel\Scout\Searchable;\n\nclass Article extends Model\n{\n    use Searchable;\n\n    public function toSearchableArray()\n    {\n        return [\n            'title'   => $this->title,\n            'content' => $this->content,\n            'vector'  => $this->embedding, // stored as JSON\n        ];\n    }\n}\n

When you call Article::search($query)->get(); Scout will forward the request to the Pinecone driver.

Integrating Pinecone as the vector database

Pinecone offers a managed, low‑latency index optimized for up to 10 million vectors per project. After signing up, create an index with 1536 dimensions (matching OpenAI’s output) and a metric of cosine similarity.

curl -X POST \
  -H "Api-Key: $PINECONE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"dimension":1536,"metric":"cosine"}' \
  https://controller.$PINECONE_ENVIRONMENT.pinecone.io/databases\n

In Laravel, a thin wrapper around Guzzle can upsert vectors:

class PineconeClient\n{\n    protected $http;\n\n    public function __construct()\n    {\n        $this->http = new Client([\n            'base_uri' => "https://{$this->env()}.pinecone.io/",\n            'headers'  => ['Api-Key' => env('PINECONE_API_KEY')],\n        ]);\n    }\n\n    protected function env()\n    {\n        return env('PINECONE_ENVIRONMENT');\n    }\n\n    public function upsert(string $index, array $vectors)\n    {\n        $this->http->post("indexes/{$index}/vectors/upsert", [\n            'json' => ['vectors' => $vectors],\n        ]);\n    }\n\n    public function query(string $index, array $vector, int $topK = 10)\n    {\n        $response = $this->http->post("indexes/{$index}/query", [\n            'json' => [\n                'vector' => $vector,\n                'topK'   => $topK,\n                'includeMetadata' => true,\n            ],\n        ]);\n        return json_decode($response->getBody(), true);\n    }\n}\n

The Scout driver simply calls upsert during model saving and query when a search is performed.

Putting it all together: a step‑by‑step workflow

1. **Create the embedding** – When a new article is saved, the OpenAIEmbeddingService generates a vector and stores it in the embedding column (JSON).
2. **Sync to Pinecone** – The model’s saved event triggers Scout’s toSearchableArray, which the custom driver upserts the vector to the Pinecone index.
3. **Search request** – The controller receives a user query, calls the embedding service to convert the query text into a vector, then asks Scout to search. Under the hood Scout forwards the vector to Pinecone, which returns the IDs of the most similar articles.
4. **Display results** – Retrieve the Laravel models by the returned IDs and render them with Blade.

public function search(Request $request, OpenAIEmbeddingService $embed, Article $model)\n{\n    $queryVector = $embed->embed($request->input('q'));\n    $ids = Article::search($queryVector)->raw()['matches'] ?? [];// Scout driver returns raw response\n    $articles = $model->whereIn('id', array_column($ids, 'id'))->get();\n    return view('search.results', compact('articles'));\n}\n

Performance tips for production Laravel deployments

• **Batch upserts** – When importing legacy content, send vectors in groups of 500 to stay under Pinecone’s request size limit and reduce HTTP overhead.
• **Cache embeddings** – Store the JSON‑encoded vector in Redis for 24‑48 hours; repeated queries for the same phrase avoid extra OpenAI calls and cut costs (each 1,000 tokens costs $0.0004).
• **Monitor latency** – Pinecone provides metrics per index. Set an alert if 95th‑percentile query time exceeds 120 ms; you can then scale the index or adjust the replica count.

Conclusion

By marrying OpenAI’s state‑of‑the‑art embeddings with Laravel Scout’s expressive API and Pinecone’s scalable vector store, you can deliver a true semantic search experience without reinventing the wheel. The approach stays fully within Laravel’s ecosystem, leverages familiar Eloquent models, and keeps the heavy lifting—vector similarity—offloaded to a managed service. Implement the steps above, monitor costs, and your application will instantly feel smarter to end users.

Author: Mahmut Sarıkaya — sarikayadev.com

Sources

OpenAI API Documentation, Pinecone Developer Guide, Laravel Scout Official Documentation