Data Analysis of Dataset_Study4.csv

Data Mining and Analytics Course

Introduction

This project analyzes a segment of the POPANE dataset, which captures psychophysiological responses of 1157 healthy young adults experiencing positive and negative emotions. Our specific focus is on 83 subjects to investigate how anxiety impacts consumer product evaluations.

The study examines how self-construal types and anxiety-driven sympathetic nervous system activation influence consumer decision-making. Understanding these emotional drivers is important for identifying how anxiety affects assimilation and differentiation needs in consumer behavior.

This project contributes to fields such as affective computing, behavioral analytics, and consumer psychology.

Dataset Overview

• 166 observations representing 83 participants under two experimental conditions
• Emotional-state labels:
  • Baseline (–1)
  • Neutral (110)
  • Fear / Anxiety (210)
• Data collection:
  • Linguistic task to prime independent or interdependent self-construal
  • Movie to elicit neutral or anxiety state
• HRV feature domains:
  • Time domain
  • Frequency domain
  • Nonlinear metrics

Methodology and Workflow

1) Library Import

Core scientific computing and machine learning tools:

• Pandas, NumPy for processing
• Matplotlib, Seaborn for visualization
• Scikit-learn for modeling and feature engineering

2) Data Loading and Preprocessing

• Loaded into Pandas DataFrame
• Checked for missing and inconsistent values
• Ensured structure compatibility for modeling
• Initial exploratory data analysis performed

Preprocessing provided a clean foundation for accurate feature engineering and classifier training.

3) Data Exploration and Visualization

• Time Domain HRV Analysis: Features such as HRV_MeanNN, HRV_SDNN, and HRV_RMSSD demonstrated distribution differences between emotional conditions. Several outliers were observed, indicating strong physiological deviations during heightened emotional responses.

• Frequency Domain HRV Analysis: Metrics including HRV_LF, HRV_HF, HRV_LFHF and others showed variation related to autonomic nervous system activity. LF/HF ratio differences were particularly informative for anxiety detection.

• Nonlinear HRV Analysis: Features such as HRV_SD1, HRV_SD2, and SampEn revealed changes in complexity and irregularity of heart dynamics depending on emotion.

• Correlation Analysis:

  • Identified redundant, highly correlated features
  • Revealed uniquely informative features that contribute to emotional state classification

Findings confirm that HRV signals contain valuable information for psychophysiological emotion prediction.

4) Data Normalization Strategy

• Purpose: account for individual physiological baselines and highlight emotional shifts
• Process:
  • Baseline–emotion differential features computed per subject
  • Emotional state labels retained
  • RobustScaler normalization applied to preserve informative outliers
• Outcome:
  • Enhanced representation of emotional effects
  • Retained critical physiological variability relevant to psychological interpretation

5–6) Feature Engineering

• Recursive Feature Elimination (RFE):

  • Random Forest Classifier used to rank predictive features
  • Top 5 features selected to reduce dimensionality
  • Improved generalization and interpretability

• Principal Component Analysis (PCA):

  • Used to analyze variance structure
  • Visualized cumulative explained variance
  • Aided in identifying optimal dimensionality

Feature engineering improved model performance and reduced the risk of overfitting.

7) Model Development and Evaluation

Two classifiers were trained and compared:

Model	Purpose
Random Forest Classifier	Robust performance with feature importance insights
Support Vector Machine (RBF kernel)	Effective with nonlinear, high-dimensional data

• Train/Test Split: 80% / 20%
• Evaluation Metrics: Accuracy; Precision, Recall, F1-score (Classification Report)

This stage validated the feasibility of predicting emotional state using psychophysiological features.

8) Hyperparameter Tuning – Random Forest

Since Random Forest demonstrated stronger results, it was fine-tuned using GridSearchCV.

• Parameters tuned: n_estimators, max_depth, min_samples_split, min_samples_leaf
• Details:
  • 5-fold cross-validation for reliable performance estimation
  • Performance visualized using confusion matrix and evaluated using classification metrics

This optimization step significantly improved model accuracy and reduced misclassification.

Conclusion

The project successfully:

• Analyzed HRV features across emotional states
• Normalized physiological responses against baseline conditions
• Identified top predictive features using RFE and PCA
• Developed and tuned a machine learning model for emotion recognition

These results provide valuable insights into how anxiety alters physiological responses relevant to consumer behavior and contribute to psychological and data mining research.