research
∙
04/06/2023
Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark
Artificial agents have traditionally been trained to maximize reward, wh...
share
research
∙
10/18/2022
How Would The Viewer Feel? Estimating Wellbeing From Video Scenarios
In recent years, deep neural networks have demonstrated increasingly str...
research
∙
12/30/2021
Towards Robustness of Neural Networks
We introduce several new datasets namely ImageNet-A/O and ImageNet-R as ...
research
∙
05/20/2021
Measuring Coding Challenge Competence With APPS
While programming is one of the most broadly applicable skills in modern...
research
∙
03/05/2021
Measuring Mathematical Problem Solving With the MATH Dataset
Many intellectual endeavors require mathematical problem solving, but th...
research
∙
09/07/2020
Measuring Massive Multitask Language Understanding
We propose a new test to measure a text model's multitask accuracy. The ...
research
∙
08/05/2020
Aligning AI With Shared Human Values
We show how to assess a language model's knowledge of basic concepts of ...
research
∙
06/29/2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
We introduce three new robustness benchmarks consisting of naturally occ...
research
∙
11/25/2019
A Benchmark for Anomaly Segmentation
Detecting out-of-distribution examples is important for safety-critical ...
research
∙
08/01/2019
DIODE: A Dense Indoor and Outdoor DEpth Dataset
We introduce DIODE, a dataset that contains thousands of diverse high re...
research
∙
07/16/2019