Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 

Repository files navigation

🧠 Natural Language Processing (NLP) Assignment

📌 Overview

This project demonstrates the implementation of multiple Natural Language Processing (NLP) techniques using Python in a Jupyter Notebook. The goal is to process raw text data (news articles, product reviews, or social media posts) and apply various NLP tasks while comparing different approaches.


🚀 Features Implemented

🔹 Text Preprocessing

  • Regex-based text cleaning (removal of URLs, hashtags, emails)
  • Tokenization-based cleaning
  • Comparison of both methods

🔹 Stemming vs Lemmatization

  • Implemented using NLTK
  • Compared accuracy and efficiency

🔹 Named Entity Recognition (NER)

  • Implemented using spaCy
  • Compared rule-based vs ML-based approaches

🔹 POS Tagging & N-grams

  • POS tagging using NLTK
  • N-gram models (Unigram, Bigram, Trigram)
  • Next word prediction analysis

🔹 Word Sense Disambiguation (WSD)

  • Dictionary-based approach using WordNet
  • Compared with ML-based methods

🔹 Word Embeddings

  • Implemented Word2Vec using Gensim
  • Discussed GloVe comparison
  • Used embeddings for text representation

🔹 Parsing Techniques

  • Context-Free Grammar (CFG)
  • CKY Parsing
  • Dependency Parsing (Subject–Verb–Object extraction)
  • Comparison with constituency parsing

🔹 Coreference Resolution

  • Basic heuristic-based pronoun resolution
  • Compared with deep learning approaches

📊 Comparisons Performed

Task Method 1 Method 2 Conclusion
Text Cleaning Regex Tokenization Tokenization is more flexible
Stemming vs Lemmatization Stemming Lemmatization Lemmatization is more accurate
NER Rule-based ML-based ML-based performs better
N-grams Unigram Bigram/Trigram Higher n improves context
Parsing CKY Earley CKY is efficient but rigid

🛠️ Technologies Used

  • Python 🐍
  • Jupyter Notebook 📓
  • NLTK
  • spaCy
  • Gensim
  • Regular Expressions (re)

📂 Project Structure

📁 NLP_Assignment
 ├── 📄 NLP_Assignment.ipynb
 ├── 📄 README.md


▶️ How to Run

  1. Clone the repository:
git clone https://github.com/your-username/NLP_Assignment.git
  1. Install dependencies:
pip install nltk spacy gensim pandas
python -m spacy download en_core_web_sm
  1. Run the notebook:
jupyter notebook

📌 Results & Insights

  • Lemmatization provides better linguistic accuracy than stemming.
  • ML-based NER significantly outperforms rule-based approaches.
  • Trigram models improve prediction accuracy but increase complexity.
  • Dependency parsing is more useful for extracting relationships than constituency parsing.

📚 Conclusion

This project covers fundamental and advanced NLP techniques and highlights the trade-offs between different approaches in terms of accuracy, efficiency, and complexity.


✨ Author

Sneha Sarraf SRM Institute of Science and Technology

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages