Much of the research on the comprehension of passive sentences has targeted healthy adults, L1 acquirers and people with aphasia. However, a topic that lacks evidence is the comprehension challenges of EFL learners facing different passive structure types. Consequently, this study investigated the comprehension difficulty of different passive structures by 186 intermediate EFL learners. The participants’ task was to read a sentence and choose a corresponding answer in a multiple-choice format via a software application designed for this study. The answers were analyzed in terms of the comprehension accuracy and the reaction time. Compared with the passive sentences, the participants needed less time comprehending active sentences and had a higher success rate. The results suggested that different passive verb types (i.e., regular/irregular, action/state, double-object/single-object, negative/affirmative, and question/statement) imposed different degrees of comprehension challenge to EFL learners; passives with regular verbs (PR) were the least challenging and passives with double-object verbs (PDO) were the most demanding structures. It was also revealed that the participants’ comprehension of different passive structures was significantly different based on their reaction times. The study’s findings may be of insight for EFL instructors and material developers to possibly invest more time for the more challenging passive structures.