A novel optimized parametric hyperbolic tangent swish activation function for 1D-CNN: application of sensor-based human activity recognition and anomaly detection

Human activity recognition (HAR) and abnormal / anomaly detection have significant applications for health monitoring in a smart environment. Abnormal / anomaly prediction during daily activities helps to indicate whether the person is healthy or having behavior issues that need assistance. HAR has been accomplished using deep learning approaches. The activation functions employed in deep learning models have a significant impact on the training model and the reliability of the model. Several activation functions are developed for deep learning models. Nevertheless, most existing activation functions suffer from the dying gradient problem and lack of utilization of large negative input values. This work proposes a novel activation function called Optimized Parametric Hyperbolic Tangent Swish (OP-Tanish), which is non-monotonicity, unbounded in both negative and positive directions, smooth variations, and introduces a higher degree of non-linearity than Relu and other state-of-art activation functions. We test this activation function by training on customized shallow 1D- Convolutional Neural Networks (CS1DCNN) for adequate recognition of human activities and anomaly detection. The contributions are compared to the state-of-the-art activation functions on benchmark datasets (UCI-HAR, PAMPA2, Opportunity, and Daphnet Gait HAR datasets; UP-FALL and Simulated Activities of Daily Living (SIMADL) anomaly datasets), namely: ReLu and its variants, SWISH, MISH, and LiSHT. The proposed OP-Tanish activation function outperforms other state-of-art activation functions with an accuracy of 99.58%, 99.58%, 95.14%, and 97.79%over UCI, Opportunity, PAMPA2, and Daphnet Gait datasets, respectively, and achieved an accuracy of 97.28% and 98% on UP-Fall and SIMADL dataset.