Subscribe
Sign in
EleutherAI GPT-Neo
p/eleutherai-gpt-neoOpen-source GPT-3
Start new thread
Aditya

GPT-J - Open-source cousin of GPT-3, everyone can use it

by•

GPT⁠⁠-⁠J-6B, a 6 billion parameter model trained on the Pile, is now available for use with our new codebase, Mesh Transformer JAX.

Add a comment

Replies

Best
Raju Akon
How to Use it?
Aditya
Hunter
@raju_akon super simple. Just visit https://6b.eleuther.ai/?ref=prod... write a prompt (the topic/reference/plot) on which you want an output, and then click on 'Run the model' 😍
Blake Hunsicker
This looks really promising. Does GPT-J have a token limit like GPT-3 does?
Aditya
Hunter
@blakehunsicker yes, it can be fine-tuned at a rate of ~5000 tokens/second, which should be sufficient for small-to-medium-size datasets. Fine tuning instructions are here: https://github.com/kingoflolz/me...
Aditya
Hunter
Yep! Doors have been OPENED 🤯 An open-source cousin of GPT-3 is here 😇 - Performs on par with 6.7B GPT-3 - Performs better and decodes faster than GPT-Neo - repo + colab + free web demo Got to know about it through Towards Data Science article: https://towardsdatascience.com/c... More details in @arankomatsuzaki's article: https://arankomatsuzaki.wordpres...
Mayank Mishra
Launching soon!
this is super good for folks to get started with GPT3 @adityavsc
Aditya
Hunter
@mishra_mayank absolutely!
Pal
Fab find @adityavsc Thanks! How safe has GPT-J been in your usage - any issues of negative sentiments, foul language or worse?
Aditya
Hunter
@pallpakk some results were definitely weird but overall, it works great! Negative sentiment, foul language, etc are context specific outputs. So if an input is negative/abusive itself, the output is bound to reinforce the same sentiment.
Pal
@adityavsc Thank you for clarifying. I'm definitely looking into this some more!
Ajeya
🔥🔥🔥🔥🔥
Nik Hazell
Amazing! Nice one for hunting this down @adityavsc!
Aditya
Hunter
@nik_hazell thank you! Anything for open-source 😉
Girdharee Saran
Useful and easy, good luck on launch
Matt Gordon
Will try this out
Swebliss
Wow this is amazing. Thank you SO much. Is there any way to DM you and ask you something? :)
Ankit Sharma
awesome bro. keep going. It is much needed. Congo 👍👍👍
Aditya
Hunter
@ankitsharmaofficial all credit goes to the open-source community :)
Nassim Abd
Great, we were waiting for this
Aditya
Hunter
@nassc for so long! Finally the wait is over :)
Dillon Peterson
WOAH!! Way to go guys! Thank y'all for putting this together, really amazing.
Aditya
Hunter
@dillon_peterson all credit goes to the wonderful open-source community :)
Fateh BENMERZOUG, Ph.D
This just blew up the door of textual content generation, awesome!
Aditya
Hunter
@fateh_benmerzoug IKR! This is literally giving super-powers to the Makers 🚀
Kelvin Zhao
Will an api of this be created?
Raphi Mbinga
🔥🔥🔥
Nick Dahlhoff
Looks really interesting
Pascal Weinberger
the world needs this :)
Patrick Hamelin
*GPT-J is just as good as GPT-3.* It is more efficient, but with more quirks. In our JPRED scores, it did better with simple TCS tasks, but lost with the more complex tasks. By removing the Jordan Algorithm: Our next proposed change to a probability model is removing the Jordan Algorithm. The Jordan Algorithm is a special procedure used for simple TCS tasks that allows for fast analysis of different sequence pairs, as well as being able to easily analyze simple n-gram (aka word) models.It is more efficient, but with more quirks. In our JPRED scores, it did better with simple TCS tasks, but lost with the more complex tasks. ...