Skip to content
This repository was archived by the owner on Sep 19, 2020. It is now read-only.

Commit d7a9bb5

Browse files
Update model-card.md
Updated from May 2020 to September 2020 version of model card.
1 parent 97920c0 commit d7a9bb5

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

model-card.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# GPT-3 Model Card
22

3-
Last updated: August 2020
3+
Last updated: September 2020
44

55
Inspired by [Model Cards for Model Reporting (Mitchell et al.)](https://arxiv.org/abs/1810.03993), we’re providing some accompanying information about the 175 billion parameter GPT-3 model.
66

@@ -9,7 +9,7 @@ Inspired by [Model Cards for Model Reporting (Mitchell et al.)](https://arxiv.or
99
GPT-3 is a Generative Pretrained Transformer or “GPT”-style autoregressive language model with 175 billion parameters. Researchers at OpenAI developed the model to help us understand how increasing the parameter count of language models can improve task-agnostic, few-shot performance. Once built, we found GPT-3 to be generally useful and thus created an API to safely offer its capabilities to the world, so others could explore them for commercial and scientific purposes.
1010

1111
### Model date
12-
May 2020
12+
September 2020
1313

1414
### Model type
1515
Language model
@@ -64,7 +64,7 @@ Interpretability & predictability: the capacity to interpret or predict how GPT-
6464

6565
High variance on novel inputs: GPT-3 is not necessarily well-calibrated in its predictions on novel inputs. This can be observed in the much higher variance in its performance as compared to that of humans on standard benchmarks.
6666

67-
Creation date of training corpora: GPT-3 was trained on a dataset created in November 2019, so has not been trained on any data more recent than that.
67+
Creation date of training corpora: The May 2020 version of GPT-3 was trained on a dataset created in November 2019, so has not been trained on any data more recent than that. The September 2020 version of the model was retrained to reflect data up to August 2020.
6868

6969
Biases: GPT-3, like all large language models trained on internet corpora, will generate stereotyped or prejudiced content. The model has the propensity to retain and magnify biases it inherited from any part of its training, from the datasets we selected to the training techniques we chose. This is concerning, since model bias could harm people in the relevant groups in different ways by entrenching existing stereotypes and producing demeaning portrayals amongst other potential harms.<sup>[[5]](#fn5)</sup> This issue is of special concern from a societal perspective, and is discussed along with other issues in the [paper](https://arxiv.org/abs/2005.14165) section on Broader Impacts.
7070

0 commit comments

Comments
 (0)