echohive42
patreon
Langchain: Auto evaluate llms and agents video files:
🕑 Added 2023-02-26 00:19:25 +0000 UTCfiles are for video: https://youtu.be/6ukUopbOr58
Langchain evaluation docs: https://langchain.readthedocs.io/en/latest/use_cases/evaluation.html miniconda: https://docs.conda.io/en/latest/miniconda.html Huggingface datasets: https://huggingface.co/docs/datasets/installation
echohive blog: https://echohive.ghost.io/
Comments
Echo Hive
Maybe try updating to the latest langchain version.
Brian Porter
I'm a bit late to the game on this one but I'm running into parsing issues when trying to run the agent_eval.py file. I will get a few observations and actions but then will hit an error like: "langchain.schema.OutputParserException: Could not parse LLM output: `I need to read the article to find the exact percentage. Action: Click on the article from the previous search`" Some fixes online suggest writing a custom parser but that seems pretty complicated (I am a novice). Anybody else run into this issue and find a way to resolve it?
Echo Hive
Yeah it should. All this is doing is evaluating the return from an llm and ground truth by getting an llm to compare the two.
Mark
This is amazing, would this work with unstructured data?
