"No one is harder on a talented person than the person themselves" - Linda Wilkinson ; "Trust your guts and don't follow the herd" ; "Validate direction not destination" ;
Showing posts with label Topic Modelling. Show all posts
Showing posts with label Topic Modelling. Show all posts

February 22, 2023

NLP, Recruitment, AI - Automated Matching :) Eightfold - Analysis

A ton of zero-shot / few shot / data labeling to customize and build models for

Some of the Key features / NLP overlaps

Custom Entity Recognition

  • Education
  • Place
  • Company 
  • Domain
  • Skills Extraction

Custom Embedding for

  • Similar Projects Search
  • Project Summarization
  • Creating custom embedding for each domain

NLP + Ranking

  • Ranking and retrieving based on location/salary/education
  • Skill Distribution / Contribution
  • Domain Extraction from Company names
Vision
  • OCR + Vision for Content Retrieval

Search / Retrieval

  • Vector database for Finetuning/indexing vector databases to search and retrieve the closest matches

From JD

  • Familiar with Language models, and transformers like BERT, GPT-3, T-5 etc.
  • Prior experience building and deploying machine learning models in production at scale
  • Familiarity with MLOps tools and pipelines (MLflow, Metaflow).
  • Integration with Workday, SuccessFactors, Taleo, PeopleSoft, iCIMS, SmartRecruiters,
  • REST APIs, microservices, data ingestion and processing systems, and distributed systems.



May 28, 2022

Topic Modelling - LDA, LSA

  • LDA stands for Latent Dirichlet Allocation, and it is a type of topic modeling algorithm
  • LDA was developed in 2003 by researchers David Blei, Andrew Ng and Michael Jordan
  • LDA is based on a Bayesian framework. This allows the model to infer topics based on observed data (words) through the use of conditional probabilities
  • The main difference between LSA and LDA is that LDA assumes that the distribution of topics in a document and the distribution of words in topics are Dirichlet distributions. LSA does not assume any distribution and therefore, leads to more opaque vector representations of topics and documents
  • Latent Semantic Analysis or Latent Semantic Indexing – Uses Singular Value Decomposition (SVD) on the Document-Term Matrix
  • In practice, LSA is much faster to train than LDA, but has lower accuracy.

Example


Keep Thinking!!!