TY - GEN
T1 - Hierarchical attention network with XGBoost for recognizing insufficiently supported argument
AU - Suhartono, Derwin
AU - Gema, Aryo Pradipta
AU - Winton, Suhendro
AU - David, Theodorus
AU - Fanany, Mohamad Ivan
AU - Arymurthy, Aniati Murni
N1 - Funding Information:
mass, strength and quality in adults in China. Methods Based on the second resurvey of China Kadoorie Biobank (CKB) in 2013-2014, logistic regression models were used to analyze the correlation of different types, number and duration of chronic diseases with low muscle mass, handgrip strength and muscle quality. Results The prevalence rate of diabetes, coronary heart disease (CHD), stroke and chronic obstructive pulmonary disease (COPD) were 9.6%, 5.8%, 3.2% and 26.8%, respectively, and 38.8% of the participants had at least one disease, and they were more likely to have low handgrip strength and low arm muscle quality (AMQ), and the longer the chronic diseases duration, the higher the risk. The ORs (95%CIs) for low handgrip strength and low AMQ in patients with 1 chronic disease for more than 10 years was 1.64 (1.42-1.90) and 1.83 (1.60-2.10), respectively. The ORs (95%CIs) for low handgrip strength were 1.26 (1.17-1.37), 1.42 (1.23-1.64) and 2.27 (1.55-3.32) and the ORs (95%CIs) for low AMQ were 1.28 (1.18-1.38), 1.67 (1.46-1.92) and 2.41(1.69-3.45), respectively, in patients with 1, 2, ≥3 chronic diseases, the correlation showed a linear trend (P for trend <0.001). Diabetes, CHD and stroke were positively correlated with low handgrip strength and low AMQ. Compared with participants without COPD, COPD patients were more likely to have low appendicular skeletal muscle mass index (ASMI), low total skeletal muscle mass index (TSMI) and low handgrip strength, and the risk was positively correlated with disease duration. Conclusions Patients with chronic diseases were more likely to have lower muscle strength and muscle quality, especially the patients with multi diseases and longer disease duration. The proportion of low handgrip strength and low AMQ was higher in patients who reported multi-prevalence and longer duration of chronic diseases. 【Key words】 Chronic disease; Muscle mass; Handgrip strength; Muscle quality Fund programs: National Natural Science Foundation of China (81941018, 91846303, 91843302, 81390540, 81390541, 81390544); National Key Research and Development Program of China (2016YFC0900500, 2016YFC0900501, 2016YFC0900504); Kadoorie Charitable Foundation in Hong Kong of China; Wellcome Trust in the UK (212946/Z/18/Z, 202922/Z/16/Z, 104085/Z/14/Z, 088158/Z/09/Z)
Publisher Copyright:
© Springer International Publishing AG 2017.
PY - 2017
Y1 - 2017
N2 - In this paper, we propose the empirical analysis of Hierarchical Attention Network (HAN) as a feature extractor that works conjointly with eXtreme Gradient Boosting (XGBoost) as the classifier to recognize insufficiently supported arguments using a publicly available dataset. Besides HAN + XGBoost, we performed experiments with several other deep learning models, such as Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), bidirectional LSTM, and bidirectional GRU. All results with the best hyper-parameters are presented. In this paper, we present the following three key findings: (1) Shallow models work significantly better than the deep models when using only a small dataset. (2) Attention mechanism can improve the deep model’s result. In average, it improves Area Under the Receiver Operating Characteristic Curve (ROC-AUC) score of Recurrent Neural Network (RNN) with a margin of 18.94%. The hierarchical attention network gave a higher ROC-AUC score by 2.25% in comparison to the non-hierarchical one. (3) The use of XGBoost as the replacement for the last fully connected layer improved the F1 macro score by 5.26%. Overall our best setting achieves 1.88% improvement compared to the state-of-the-art result.
AB - In this paper, we propose the empirical analysis of Hierarchical Attention Network (HAN) as a feature extractor that works conjointly with eXtreme Gradient Boosting (XGBoost) as the classifier to recognize insufficiently supported arguments using a publicly available dataset. Besides HAN + XGBoost, we performed experiments with several other deep learning models, such as Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), bidirectional LSTM, and bidirectional GRU. All results with the best hyper-parameters are presented. In this paper, we present the following three key findings: (1) Shallow models work significantly better than the deep models when using only a small dataset. (2) Attention mechanism can improve the deep model’s result. In average, it improves Area Under the Receiver Operating Characteristic Curve (ROC-AUC) score of Recurrent Neural Network (RNN) with a margin of 18.94%. The hierarchical attention network gave a higher ROC-AUC score by 2.25% in comparison to the non-hierarchical one. (3) The use of XGBoost as the replacement for the last fully connected layer improved the F1 macro score by 5.26%. Overall our best setting achieves 1.88% improvement compared to the state-of-the-art result.
KW - Deep learning
KW - Hierarchical Attention Network
KW - Insufficiently supported argument
KW - Shallow learning
KW - XGBoost
UR - http://www.scopus.com/inward/record.url?scp=85034266613&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-69456-6_15
DO - 10.1007/978-3-319-69456-6_15
M3 - Conference contribution
AN - SCOPUS:85034266613
SN - 9783319694559
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 174
EP - 188
BT - Multi-disciplinary Trends in Artificial Intelligence - 11th International Workshop, MIWAI 2017, Proceedings
A2 - Phon-Amnuaisuk, Somnuk
A2 - Ang, Swee-Peng
A2 - Lee, Soo-Young
PB - Springer Verlag
T2 - 11th Multi-disciplinary International Workshop on Artificial Intelligence, MIWAI 2017
Y2 - 20 November 2017 through 22 November 2017
ER -