The increasing societal impact of AI
- GPT3 is a big step forward
- Refinement of answer based on versions
- Graph of human performance vs machines
- Benchmarks for image, speech, question and answers
- Model size increases in recent years
- 2018 - 100 Million Parameters
- GPT3 - 175 Billion parameters
What can be contribute
- Model Testing
- Model Fairness
- Model Implementation in production
- Transfer Learning to customize to domains
In-Context Learning, Help model learn from prompt
- Rise of in context learning
- Standard Supervision vs In-Context
- Earlier labeled datasets
- Few shots in context learning
- Mechanism how it works
- Transformer Architecture
- Word Embedding
- Positional Encoding
- Attention mechanism
- Feedforward
- Regularization layers
- Self Supervision
- co-occurrence patterns
- Distributional learning
- High probability of attested sequences
Large scale pre-training
- word2vec
- glove
- Elmo contextual representation
- Bert
- GPT
Human Feedback in OpenAI Models
- Instruct models
- Finetuned ranked by human feedback
- Return to familiar formats of results
- Instruct is best in class
- Language and Retrieval Models
- Retrieval mechanism for relevant evidence
- Knowledge store / Updateability
- Trustability - references / How it arrived at the answer
Retrieval Augmented NLP
- LLM strategy
- Scoring the documents
- Representations scored
- LM for reader / generator to synthesize
- Data indexed and updated
- Data tied to model, When data changes model also updated
- Models can communicate
- Retriever / LLM
- Index / Scores
- Filter Generations
- Reproducing the distribution
- Retrieval Options
- Retrieval Augmented options
- Model Explainability
- Reliability
- Safety
- Trust
Predictions Finally :)
Ref - Link
Retrieval based approach is more explainable, traceable but can be tracked for updates
Hybrid approaches will continue to evolve as we get into more manageable mainstream models.
Keep Exploring!!!
No comments:
Post a Comment