Data Exploration and Profiling

Profile categorical and nominal variables for frequency distributions, cardinality, encoding issues, and rare categories. Expert in label consistency, cardinality reduction, and encoding strategy selection.

Profile dataset schemas, infer data types, detect type mismatches, and generate data dictionaries. Expert in schema validation, inferred vs. declared type reconciliation, and metadata documentation.

Profile dataset quality across completeness, consistency, validity, uniqueness, and timeliness dimensions. Generates quality scorecards, issue inventories, and remediation recommendations.

Perform structured exploratory data analysis to uncover distributions, outliers, correlations, and patterns. Generates EDA reports, visualizations, and statistical summaries in Python or R.

Profile and explore high-dimensional datasets using PCA, t-SNE, UMAP, and feature variance analysis. Expert in dimensionality assessment, curse of dimensionality diagnosis, and structure visualization.

Diagnose missing data mechanisms (MCAR, MAR, MNAR) and design appropriate imputation strategies. Expert in missingness visualization, Little's MCAR test, and multiple imputation methods.

Explore relationships between multiple variables using correlation matrices, pair plots, VIF analysis, and mutual information. Expert in multicollinearity detection, non-linear associations, and mixed-type correlation.

Detect, classify, and profile outliers in univariate and multivariate datasets. Expert in IQR, z-score, Isolation Forest, LOF, and DBSCAN-based anomaly detection with business impact assessment.

Explore and profile time series data for trends, seasonality, stationarity, and anomalies. Expert in ACF/PACF analysis, decomposition, irregularity detection, and temporal data quality assessment.

Characterize single-variable distributions with statistical tests, goodness-of-fit analysis, and transformation recommendations. Expert in normality testing, skewness correction, and distribution fitting.