 We proposed a novel hierarchical recurrent neural network, HRNN, to capture the temporal dynamics of electroencephalogram, e.g. signals recorded from the scalp surface. This HRNN is capable of encoding all EEG signals simultaneously and modeling them with a temporal manner to predict which visual event elicits an electroencephalogram, e.g. Furthermore, we apply dynamic adversarial perturbation to create adversarial examples to enhance the model performance. Experiments are conducted on one published visual ERP-based BCI task with 15 subjects and three different auditory workload states. Results demonstrate that our hierarchical method can effectively model the long sequence EEG raw data, outperform the baseline on most conditions, including cross-state and cross-subject conditions. This article was authored by Z&I, Jeming Su, Yue Wu, and others.