Automated Feature Engineering (AFE) has been successful in generating and selecting optimal feature sets for downstream tasks. However, current AFE methods primarily focus on improving the effectiveness of the produced features, neglecting the issue of low efficiency for large-scale deployment. In this work, a generic framework is proposed to enhance the efficiency of AFE. The framework constructs an AFE pipeline based on a reinforcement learning setting. Each feature is assigned an agent to perform feature transformation and selection, with the evaluation score of the produced features in downstream tasks serving as the reward to update the policy. The efficiency of AFE is improved from two perspectives. Firstly, a Feature Pre-Evaluation (FPE) Model is developed to reduce the sample size and feature size which are two main factors undermining feature evaluation efficiency. This model helps in reducing computational resources required for evaluating features. Secondly, a two-stage policy training strategy is devised by running FPE on the pre-evaluation task as initialization of the policy. This approach avoids training the policy from scratch and further improves computational efficiency. Comprehensive experiments are conducted on 36 datasets involving both classification and regression tasks. The results demonstrate an average performance improvement of 2.9% compared to state-of-the-art AFE methods while achieving 2x higher computational efficiency. This work not only improves effectiveness but also addresses low-efficiency issues in large-scale deployment of AFE methods by introducing a generic framework that incorporates reinforcement learning and pre-evaluation techniques providing a more efficient approach for automated feature engineering tasks.
- - Automated Feature Engineering (AFE) successful in generating and selecting optimal feature sets
- - Current AFE methods focus on effectiveness, neglecting efficiency for large-scale deployment
- - Proposed generic framework to enhance efficiency of AFE
- - Framework constructs AFE pipeline based on reinforcement learning setting
- - Each feature assigned an agent for transformation and selection
- - Evaluation score of produced features serves as reward to update policy
- - Efficiency improved from two perspectives:
- - Feature Pre-Evaluation (FPE) Model reduces sample size and feature size for efficient evaluation
- - Two-stage policy training strategy using FPE as initialization to improve computational efficiency
- - Comprehensive experiments conducted on 36 datasets with classification and regression tasks
- - Results show average performance improvement of 2.9% compared to state-of-the-art AFE methods
- - Achieves 2x higher computational efficiency
- - Generic framework addresses low-efficiency issues in large-scale deployment of AFE methods
Automated Feature Engineering (AFE) is a way to create and choose the best features for a task. It can be very effective but not always efficient for big projects. A proposed framework aims to make AFE more efficient by using reinforcement learning. Each feature gets its own agent to transform and select it. The framework also uses a model called Feature Pre-Evaluation (FPE) to make the evaluation process faster. Experiments show that this framework improves performance by 2.9% compared to other methods and is twice as efficient."
Definitions- Automated Feature Engineering (AFE): Using technology to automatically create and choose the best features for a task.
- Efficiency: How well something works without wasting time or resources.
- Reinforcement learning: A type of machine learning where an algorithm learns through trial and error, getting rewards for good actions.
- Feature: A characteristic or property of something that can be used in analysis or decision-making.
- Evaluation: The process of judging how good something is based on certain criteria.
- Framework: A structure or plan that helps organize and guide a project or task.
- Computational efficiency: How quickly and effectively a computer program can perform calculations or tasks.
- State-of-the-art: The most advanced or up-to-date technology or method currently available.
Automated Feature Engineering: A Reinforcement Learning-Based Framework for Improved Efficiency
The development of automated feature engineering (AFE) methods has been a major breakthrough in the field of machine learning, allowing for the generation and selection of optimal feature sets for downstream tasks. However, current AFE methods primarily focus on improving the effectiveness of produced features while neglecting the issue of low efficiency when it comes to large-scale deployment. To address this problem, researchers have proposed a generic framework that enhances the efficiency of AFE by incorporating reinforcement learning and pre-evaluation techniques.
Background
In recent years, automated feature engineering (AFE) has become increasingly popular due to its ability to generate and select optimal feature sets for downstream tasks such as classification or regression. Current AFE methods are effective in producing high quality features but lack efficient implementation when it comes to large-scale deployment. This is because they require significant computational resources which can be prohibitively expensive or time consuming in certain scenarios. Therefore, there is an urgent need to develop an efficient approach that can improve both effectiveness and efficiency simultaneously when dealing with large datasets.
Proposed Framework
To address this challenge, researchers have proposed a generic framework based on reinforcement learning (RL) setting that improves both effectiveness and efficiency in AFE tasks. The framework consists of two main components: a Feature Pre-Evaluation (FPE) model and a two-stage policy training strategy using RL techniques.
The FPE model reduces sample size and feature size which are two main factors undermining evaluation efficiency by providing an initial estimation of each feature’s performance before full evaluation takes place. This helps reduce computational resources required for evaluating features without sacrificing accuracy significantly compared with traditional approaches where all samples must be evaluated before selecting optimal featuresets.
The second component is a two-stage policy training strategy which uses RL techniques to update policies based on rewards from downstream tasks such as classification or regression problems after each iteration cycle during training process.. In this approach, FPE serves as initialization step so that policies do not need to start from scratch every time they are trained resulting in further improved computational efficiency compared with other existing approaches where policies must be trained from scratch every time they are used..
Experimental Results
Comprehensive experiments were conducted on 36 datasets involving both classification and regression tasks using state-of-the art AFE methods as baselines for comparison purposes . The results showed an average performance improvement of 2.9% compared with baseline models while achieving 2x higher computational efficiency thanks to the incorporation of FPE model into the proposed framework . These results demonstrate how combining reinforcement learning techniques with pre-evaluation strategies can lead to more efficient implementations while still maintaining good performance levels across various types of datasets .
Conclusion
This work presents a generic framework based on reinforcement learning setting that enhances both effectiveness and efficiency in automated feature engineering tasks by introducing new pre-evaluation strategies combined with policy optimization techniques . Comprehensive experiments show promising results demonstrating improved performance levels along with increased computational speed , making this approach suitable for large scale deployments requiring fast response times while still producing high quality featuresets .