An experiment is a collection of logged events, such as model inputs and outputs, which represent a snapshot of your application at a particular point in time. An experiment is meant to capture more than just the model you use, and includes the data you use to test, pre- and post- processing code, comparison metrics (scores), and any other metadata you want to include.
Experiments are associated with a project, and two experiments are meant to be easily comparable via
inputs. You can change the attributes of the experiments in a project (e.g. scoring functions)
over time, simply by changing what you log.
You should not create
Experiment objects directly. Instead, use the
• new Experiment(
Finish the experiment and return its id. After calling close, you may not invoke any further methods on the experiment object.
Will be invoked automatically if the experiment is wrapped in a callback passed to
The experiment id.
Log a single event to the experiment. The event will be batched and uploaded behind the scenes.
|The event to log.|
Create a new toplevel span. The name parameter is optional and defaults to "root".
Span.startSpan for full details.
Summarize the experiment, including the scores (compared to the closest reference experiment) and metadata.
|Options for summarizing the experiment.|
|The experiment to compare against. If None, the most recent experiment on the origin's main branch will be used.|
|Whether to summarize the scores. If False, only the metadata will be returned.|
A summary of the experiment, including the scores (compared to the closest reference experiment) and metadata.
Experiment.startSpan, which passes the initialized
Span it to the given callback and ends it afterwards. See
Span.traced for full details.