Commits on Source 25
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-
Stergios Papadopoulos authored
-Improved MyDoc class. -Created chunk_docs and load_docs methods in Embedder class. -Created abstractions on Embedder class.
-
Stergios Papadopoulos authored
-Added __repr__ method in MyDoc class -Added get_chunks on Embedder -Tested Embedder and it works
-
Stergios Papadopoulos authored
_Added _get_embedding_gpt method _Added _vectors_generator
-
Stergios Papadopoulos authored
-Getting error in the vectorize method.
-
Stergios Papadopoulos authored
-Improved vectorize method -Checked if above work
-
Stergios Papadopoulos authored
-changed collection_exists to _get_collection and _get_collection_error -Repair load_docs color loading -Added search_similar but needs fixing i must create pickle saving class for saving embedding methods related to colections -Must set search_similar to class method
-
Stergios Papadopoulos authored
- Fixed _get_collection and _get_collection_error so that it adds same embedding function as the one used to embed data. - Added _add_title_to_chunks at mydoc module so that better results can be retrieved.
-
Stergios Papadopoulos authored
- Added visualize method for visualizing vectodb
-
Stergios Papadopoulos authored
- Added generator module that generates answers to user's questions. In the curent state it works in the command line. - Future work is to further clear chunks (keywords and unnecessary text should gone) and to create the UI.
-
Stergios Papadopoulos authored
- Future work: only save at collection the user question not the relevant context to save context window.
-
Stergios Papadopoulos authored
- Modified generate_answer() to be compatible with the ui (streaming). Also in the conversation only the LLM's answer is saved and later provided to the LLM for context window savings. - Created uicontroller module that can create a simple ui which can be shared.
-
Stergios Papadopoulos authored
- Added the option to change the n_results fro, the ui module. - Added doc_path metadata to chunks. - improved get_chunks in embedder module so that it returns chunks from specified collection. - Improved the cleaning method of chunks.
-
Stergios Papadopoulos authored
-mydoc: added _clear_pages so that pages have cleared text, this methods gets executed at constructor. - Added tester.py, right now only test set generation and visualization is available.
-
Stergios Papadopoulos authored
- added generate_answer_structured method that produces structured output for the evaluation process tester.py: -added _query_generator method that creates the generator's responses and contexts of the test set questions. -added _create_eval_set method that creates the evaluation set from the accepted test set. -added _get_metrics method that turn the given metrics as strings and returns the object representation of them. -improved generate_test_data method so that it saves them in pickle file. -renamed visualize_dataset method to upload_dataset. -added accept_test_set method that creates a Testset object from the testset.json (approved test set) and saves it in pickle. -added to_csv method that saves in csv form a given dataset. -added test classmethod that evaluates a given generator on given metrics and creates an evaluated data set. in current state it can load an existing evaluation set or create one from an accepted test set. It also uploades and saves the evaluated data set
-
Stergios Papadopoulos authored
- Improved system prompt. - Says hello when get to app. - Gave choice weather or not to stream the answer. - Created non stream answer method. - added generate structured answer method. - Made audio generation faster with threads. mydoc.py: