Chat with your data using OpenAI, Pinecone, Airbyte and Langchain

Abstraction: RAG pipeline tutorial with Airbyte and Pinecone

Key points:

• Tutorial builds a connector-development Slack bot that answers from GitHub issues, scraped docs (via Apify), and Slack messages, all synced into Pinecone.
• Airbyte's vector database destination handles chunking, embedding, and indexing automatically; supports Pinecone, Weaviate, Milvus, Qdrant, Chroma.
• Concrete settings: Pinecone index dimension 1536 (OpenAI embedding size), chunk size 1000 tokens (~4KB); incremental sync only re-embeds changed records.
• Uses LangChain's RetrievalQA chain (chain_type="stuff") with OpenAIEmbeddings and OpenAI(temperature=0); default retrieves top 5 documents.
• Improvements shown: custom prompt template setting context, per-document formatting (ConnectorDevelopmentPrompt) that injects GitHub issue number/state and source type so answers cite provenance and flag open issues.
• Slack integration via slack_sdk SocketModeClient responding to app_mention events.

Connections: Langchain · Openai · Pinecone · Airbyte · Retrieval Augmented Generation · Vector Database

Source: https://blog.langchain.dev/chat-with-your-data-using-openai-pinecone-airbyte-langchain/