# Toward Efficient Automated Feature Engineering

**Authors:** Kafeng Wang, Pengyang Wang, Chengzhong xu  
**arXiv:** [2212.13152](https://arxiv.org/abs/2212.13152)  
**Date:** 2022-12-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.13152v1/

## Key points

- Automated Feature Engineering (AFE) successful in generating and selecting optimal feature sets
- Current AFE methods focus on effectiveness, neglecting efficiency for large-scale deployment
- Proposed generic framework to enhance efficiency of AFE
- Framework constructs AFE pipeline based on reinforcement learning setting
- Each feature assigned an agent for transformation and selection
- Evaluation score of produced features serves as reward to update policy
- Feature Pre-Evaluation (FPE) Model reduces sample size and feature size for efficient evaluation
- Two-stage policy training strategy using FPE as initialization to improve computational efficiency
- Comprehensive experiments conducted on 36 datasets with classification and regression tasks
- Results show average performance improvement of 2.9% compared to state-of-the-art AFE methods
- Achieves 2x higher computational efficiency
- Generic framework addresses low-efficiency issues in large-scale deployment of AFE methods

## Summary

Automated Feature Engineering (AFE) has been successful in generating and selecting optimal feature sets for downstream tasks. However, current AFE methods primarily focus on improving the effectiveness of the produced features, neglecting the issue of low efficiency for large-scale deployment. In this work, a generic framework is proposed to enhance the efficiency of AFE. The framework constructs an AFE pipeline based on a reinforcement learning setting. Each feature is assigned an agent to perform feature transformation and selection, with the evaluation score of the produced features in downstream tasks serving as the reward to update the policy. The efficiency of AFE is improved from two perspectives. Firstly, a Feature Pre-Evaluation (FPE) Model is developed to reduce the sample size and feature size which are two main factors undermining feature evaluation efficiency. This model helps in reducing computational resources required for evaluating features. Secondly, a two-stage policy training strategy is devised by running FPE on the pre-evaluation task as initialization of the policy. This approach avoids training the policy from scratch and further improves computational efficiency. Comprehensive experiments are conducted on 36 datasets involving both classification and regression tasks. The results demonstrate an average performance improvement of 2.9% compared to state-of-the-art AFE methods while achieving 2x higher computational efficiency. This work not only improves effectiveness but also addresses low-efficiency issues in large-scale deployment of AFE methods by introducing a generic framework that incorporates reinforcement learning and pre-evaluation techniques providing a more efficient approach for automated feature engineering tasks.