"This scene" and "other scene" do refer to the incoming and outgoing scenes, but not as you suspect (respectively).
Since both the Incoming and Outgoing scenes each need to select a rule; and consider the fact that the incoming and outgoing scenes may infact be based off of different scenes; they each iterate through the rules in their own scenes to find the correct rule.
Consider scene A is online, and Scene B is coming online. When Scene A (online) looks for a rule to choose an outgoing animation, it searches the rules in Scene A. In this case "this scene" refers to Scene A and is online (going offline); and "other scene" refers to the incoming scene (Scene B).
Meanwhile, Scene B is looking for a rule to select an incoming transition. So it looks through the rules in Scene B, and while doing this, "this scene" refers to Scene B, while "other scene" refers to Scene A. So it is exactly opposite of the first case.
So, you always need to consider "this scene" and "other scene" both in terms of the scene going online and the one coming off; because they are opposite of each other.
#XPression