First draft of concept pages #27088

rlancemartin · 2024-10-03T23:11:29Z

No description provided.

vercel · 2024-10-03T23:11:30Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 30 unresolved ✅ 1 resolved	Oct 10, 2024 7:59pm

eyurtsev · 2024-10-04T14:55:32Z

docs/docs/concepts.mdx

@@ -522,120 +522,6 @@ for modifying **multiple** key-value pairs at once:

 For key-value store implementations, see [this section](/docs/integrations/stores/).

-### Tools


@rlancemartin shall we keep the section headings everywhere to maintain a glossary on the main concepts page?

We can remove the beefy content, but maintain headings so no links break

eyurtsev · 2024-10-04T14:57:38Z

docs/docs/concepts/tool_calling.mdx

+
+Many AI applications interact directly with humans (e.g., chatbots). In these cases, it is approrpiate for models to respond in natural langague. 
+
+But what about cases where we want a model to interact *directly* with another system (e.g., a databases or an API)?


another vs. additional or some other paraphrasing to mean that the AI can interact with the human AND an external system

eyurtsev · 2024-10-04T15:01:37Z

docs/docs/concepts/tool_calling.mdx

+
+`ChatModel.bind_tools()` is a method for specifying which tools are available for a model to call. 
+
+If a model has been initialized as `llm` without tools, we can bind tools to it as a list:


Do we need to be more precise about chat model vs. llm terminology? We use them interchangeably in docs, but can be confusing?

We could also opt for favoring model?

this is good point that i'm also wary of --
1/ maybe we defer to model by default
2/ conceptual guide on llm and chat model call out each specifically

eyurtsev · 2024-10-04T15:05:51Z

docs/docs/concepts/tool_calling.mdx

+
+For more details on usage, see our [how-to guides](docs/how_to/#tools)!
+
+## Common patterns 


This section is a nice addition, not sure if i've seen this discussion before.

Ya -- we use this same framing in the course.

eyurtsev · 2024-10-04T15:11:56Z

docs/docs/concepts/tool_calling.mdx

@@ -0,0 +1,133 @@
+# Tool Calling


We need a URL that we can link to that explains what a Tool is / brief example of definition/ brief example of how it can be used.

I think we need another page for tools specifically, or else retitle this page to `# Tools and have a tool calling section somewhere within it?

Some of the questions that the conceptual guide should address:

What is a tool?

Where is the tool getting executed? (maybe does a tool code need to be executed?)

How does one define a tool? (We have 4-5 ways of doing it); e.g., @tool decorator, StructuredTool.from_function..... We don't need to do it in detail, but can link to the how-to guide, but just explain high level what the differences are.

that's true. let me try to add as a section on this page.

also noted in Slack discussion. this is a good thing to flesh out --
https://langchain.slack.com/archives/C04GWPE38LV/p1728407895162059

eyurtsev · 2024-10-04T15:12:50Z

docs/docs/concepts/structured_output.mdx

+
+## Motivation 
+
+While many AI applications, such as chatbots, typically respond in natural language, there are scenarios where we need models to output in a structured format. 


maybe this?

Suggested change

While many AI applications, such as chatbots, typically respond in natural language, there are scenarios where we need models to output in a structured format.

While many AI applications, such as chatbots, typically respond in natural language, there are scenarios where we need models to output a response matching a specific structured format.

eyurtsev · 2024-10-04T15:30:41Z

docs/docs/concepts/retrieval_unstructured.mdx

+
+## Indexing Strategies
+
+Documents needed to be indexed, frequently using embedding models to [compress the semantic information in documents to fixed-size vectors](/docs/concepts/#embedding-models).


Has the term "Documents" been defined yet?

Should we use the more general term of "Content" or something along those lines so it's clear that conceptually some ideas carry to other modalities?

eyurtsev · 2024-10-04T15:34:39Z

docs/docs/concepts/retrieval_unstructured.mdx

+
+But chunk size and chunk number can be difficult to set and affect results if they do not provide full context for the LLM to answer a question. 
+
+Furthermore, LLMs are increasingly capable of processing millions of tokens. 


Latency is important for many applications.

Pulling in the content of the top 10 search hits can add significant latency.

In an extreme case, all 10 hits come from different documents.

In many domains, document length can run into the hundreds of pages (i've dozens if not hundreds of thousand page long documents in finance)

Even if the llm context window were unlimited (and could incorporate information from the context perfectly), implementation details still introduce a tradeoff in terms of latency.

Is a user willing to wait for an extra 5-60 seconds to get an answer from a chat system? This ends up being very use case dependent.

eyurtsev · 2024-10-04T15:38:45Z

docs/docs/concepts/retrieval_unstructured.mdx

+
+In some cases, irrelevant or redundant content can dilute the semantic usefulness of the embedding.
+
+[ColBERT](https://docs.google.com/presentation/d/1IRhAdGjIevrrotdplHNcc4aXgIYyKamUKTWtB3m3aMU/edit?usp=sharing) is an interesting approach to address this with a higher granularity embeddings: (1) produce a contextually influenced embedding for each token in the document and query, (2) score similarity between each query token and all document tokens, (3) take the max, (4) do this for all query tokens, and (5) take the sum of the max scores (in step 3) for all query tokens to get a query-document similarity score; this token-wise scoring can yield strong results. 


This section is describing how to implement the algorithm, but it's not explaining what the algorithm does at a conceptual level. I think most readers will gloss over this description as a result since it's not conceptual.

It would be helpful to have a sentence in the spirit of: a variation that generates multiple embeddings for the query, and retrieves closest documents to any of these embeddings

eyurtsev · 2024-10-04T15:47:37Z

docs/docs/concepts/retrieval_semi_structured.mdx

+
+Language models (LLMs) are trained on vast but fixed datasets, which limits their ability to access up-to-date or domain-specific information. 
+
+To enhance their performance on specific tasks, we can augment their knowledge using retrieval systems.


"retrieval systems" => external information (see other related comment)

eyurtsev · 2024-10-04T15:49:14Z

docs/docs/concepts/tool_calling.mdx

+In LangChain, any function can be bound as a tool.
+
+```python
+def multiply(a: int, b: int) -> int:


I believe we usually need a @tool decorator (since some code probably assumes that the tool is a runnable) at invocation time

Suggested change

def multiply(a: int, b: int) -> int:

def multiply(a: int, b: int) -> int:

First draft of concept pages

5f613d1

vercel bot had a problem deploying to Preview October 3, 2024 23:20 Failure

eyurtsev reviewed Oct 4, 2024

View reviewed changes

Revert changes to concepts doc

f3c551e

vercel bot had a problem deploying to Preview October 4, 2024 16:54 Failure

Merge branch 'concept_docs' into rlm/concept_docs

3902dda

vercel bot had a problem deploying to Preview October 7, 2024 19:28 Failure

rlancemartin added 2 commits October 8, 2024 11:52

Flesh out tool calling

ace6dbd

Flesh out tool calling

9206419

vercel bot had a problem deploying to Preview October 8, 2024 19:01 Failure

Update tool calling conepts

8d962fe

vercel bot deployed to Preview October 10, 2024 18:24 View deployment

fmt

4f89c2c

vercel bot deployed to Preview October 10, 2024 18:58 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

First draft of concept pages #27088

First draft of concept pages #27088

rlancemartin commented Oct 3, 2024 •

edited

Loading

vercel bot commented Oct 3, 2024 •

edited

Loading

eyurtsev Oct 4, 2024

eyurtsev Oct 4, 2024

eyurtsev Oct 4, 2024

rlancemartin Oct 7, 2024

eyurtsev Oct 4, 2024

rlancemartin Oct 7, 2024

eyurtsev Oct 4, 2024

rlancemartin Oct 8, 2024

rlancemartin Oct 8, 2024

eyurtsev Oct 4, 2024

eyurtsev Oct 4, 2024

eyurtsev Oct 4, 2024

eyurtsev Oct 4, 2024

eyurtsev Oct 4, 2024

eyurtsev Oct 4, 2024

		@@ -522,120 +522,6 @@ for modifying multiple key-value pairs at once:

		For key-value store implementations, see [this section](/docs/integrations/stores/).

		### Tools


		Many AI applications interact directly with humans (e.g., chatbots). In these cases, it is approrpiate for models to respond in natural langague.

		But what about cases where we want a model to interact directly with another system (e.g., a databases or an API)?


		`ChatModel.bind_tools()` is a method for specifying which tools are available for a model to call.

		If a model has been initialized as `llm` without tools, we can bind tools to it as a list:


		For more details on usage, see our [how-to guides](docs/how_to/#tools)!

		## Common patterns


		## Motivation

		While many AI applications, such as chatbots, typically respond in natural language, there are scenarios where we need models to output in a structured format.


		## Indexing Strategies

		Documents needed to be indexed, frequently using embedding models to [compress the semantic information in documents to fixed-size vectors](/docs/concepts/#embedding-models).


		But chunk size and chunk number can be difficult to set and affect results if they do not provide full context for the LLM to answer a question.

		Furthermore, LLMs are increasingly capable of processing millions of tokens.


		In some cases, irrelevant or redundant content can dilute the semantic usefulness of the embedding.

		[ColBERT](https://docs.google.com/presentation/d/1IRhAdGjIevrrotdplHNcc4aXgIYyKamUKTWtB3m3aMU/edit?usp=sharing) is an interesting approach to address this with a higher granularity embeddings: (1) produce a contextually influenced embedding for each token in the document and query, (2) score similarity between each query token and all document tokens, (3) take the max, (4) do this for all query tokens, and (5) take the sum of the max scores (in step 3) for all query tokens to get a query-document similarity score; this token-wise scoring can yield strong results.


		Language models (LLMs) are trained on vast but fixed datasets, which limits their ability to access up-to-date or domain-specific information.

		To enhance their performance on specific tasks, we can augment their knowledge using retrieval systems.

	def multiply(a: int, b: int) -> int:
	def multiply(a: int, b: int) -> int:

First draft of concept pages #27088

Are you sure you want to change the base?

First draft of concept pages #27088

Conversation

rlancemartin commented Oct 3, 2024 • edited Loading

vercel bot commented Oct 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rlancemartin commented Oct 3, 2024 •

edited

Loading

vercel bot commented Oct 3, 2024 •

edited

Loading