generator.py:
- added generate_answer_structured method that produces structured output for the evaluation process tester.py: -added _query_generator method that creates the generator's responses and contexts of the test set questions. -added _create_eval_set method that creates the evaluation set from the accepted test set. -added _get_metrics method that turn the given metrics as strings and returns the object representation of them. -improved generate_test_data method so that it saves them in pickle file. -renamed visualize_dataset method to upload_dataset. -added accept_test_set method that creates a Testset object from the testset.json (approved test set) and saves it in pickle. -added to_csv method that saves in csv form a given dataset. -added test classmethod that evaluates a given generator on given metrics and creates an evaluated data set. in current state it can load an existing evaluation set or create one from an accepted test set. It also uploades and saves the evaluated data set
Loading
Please register or sign in to comment