- Model used: GPT-3.5-turbo-1106
- Platform: OpenAI Playground
- Source used to obtain historical data.
First attempt or Zero Shot
This concept of prompt It usually does not yield a correct result, demonstrating that LLMs tend to “hallucinate” when they do not have information available in their context or when they require mathematical operations.
prompt structured as First Attempt:
At the time of Francisco I Madero’s death, how old was the mother of whoever was the king of Spain at that time?
LLM Response:
The mother of the king of Spain at the time, Alfonso XIII, was called María Cristina of Habsburg-Lorraine. At the time of Francisco I. Madero’s death in 1913, María Cristina of Habsburg-Lorraine was 64 years old.
Chain of Thought: Chain of Thought
By making a small change to the promptlike asking the model to go step by step in the process of generating a response, we can see how the behavior changes and also the result. In this case the response generated is not correct either, however these types of instructions tend to have much better results with more advanced models.
prompt for chain of thought:
Think step by step:
At the time of the death of Francisco I Madero, how old was the mother of who was at that time the king of Spain
LLM Response:
To answer that question, we must first remember the date of Francisco I Madero’s death. Madero was assassinated on February 22, 1913. From this date we need to calculate the age of the mother of the king of Spain at that time.
By doing some quick calculations, assuming that the mother of the king of Spain was at least 20 years old when she gave birth to her son, we could estimate that the mother of the king of Spain was probably at least 50 years old at the time of Madero’s death. .
However, it is important to note that this calculation is a rough estimate and the age of the King of Spain’s mother in 1913 needs to be confirmed to provide an accurate answer.
Generation increased by recovery
When we add data as context so that the model has it available, it can perform a calculation more easily and accurately. In this case, we arrive at the correct answer:
prompt for generation augmented by recovery:
-Francisco I. Madero: born October 30, 1873, died February 22, 1913
-Reign of Alfonso XIII, King of Spain: began on May 17, 1886, ended on April 14, 1931
-María Cristina de Habsburgo-Lorena, mother of Alfonso XIII: born July 21, 1858, died February 6, 1929
At the time of the death of Francisco I Madero, how old was the mother of who was at that time the king of Spain
LL Response:
At the time of Francisco I. Madero’s death, the mother of who was at that time the king of Spain was 55 years old.
Tools – Tools
This last example is an oversimplification of what we could call integrations with external tools and services. In this case it serves to exemplify how we can instruct the model to have its results structured in a specific format so that they can be used with other services (APIs).
Structured instructions (context, tools and prompt):
Context:
-Francisco I. Madero: born October 30, 1873, died February 22, 1913 – Reign of Alfonso XIII, King of Spain: began May 17, 1886, ended April 14, 1931 – María Cristina de Habsburg -Lorena, mother of Alfonso
Tools:
-calculator: This tool accepts math expressions and returns their result.
-search: This tool accepts search engine queries and returns the first search result.
If you want to use tools to arrive at the answer, output the list of tools and inputs in CSV format, with this header row `tool,input`
prompt:
Think step by step.
At the time of Francisco I Madero’s death, how old was the mother of whoever was the king of Spain that year.
LLM Response:
-tool,input
-search,Maria Christina of Habsburg-Lorraine age on 22 February 1913
-calculator,1913 – 1858
New generations of models, such as GPT-4o and higher, have integrated more advanced reasoning capabilities and the results we observe tend to be correct with simpler techniques. However, this example is relevant because in more complex scenarios and in implementations that require access to private data or specific processes, the models are not capable of offering an immediate solution. For this reason, it is important to understand how intelligent applications are built, to know what communication and guidance mechanisms are appropriate to obtain the expected results.
#Dissect #prompts
