Our EA Manager must handle more than 100 simultaneous intentions, while determining the import of a dozen or more new facts a second and checking alert histories for redundancy. It was not clear that our system could do all this and still alert the user within 5 seconds of a new fact arriving, as required by our users. We tested the EA on both scenarios to determine if it met these requirements. In real time, the EA generated alerts in less than 2 seconds from the receipt of a new fact. We found that the EA can not only keep up, but can run at between 10x and 20x real time. (There may be anomalous schedule alerts because of granularity issues at high time expansion rates.) Thus, current data rates are not close to stressing the system - at 10x real time we are processing an average of 24 events per second in our 90-minute simulation, which is double our design requirement of a dozen events per second. We did not determine the multiple at which degradation would occur because it is difficult to detect degradation in such a complex system. We did establish that the EA, using 100-meter map granularity, is easily sufficient for plan monitoring with SAIM data rates (running on both Sun Ultra 60s under Solaris and Pentium-based machines under Linux).2
Prior to implementation of the EA, we did a performance evaluation of PRS to determine if it could handle the input data rates required by the EA. We briefly describe our results as many other reactive control systems are based on PRS, e.g., UM-PRS [10]. We found that it could not handle more than 12 facts per second without unacceptably long delays, using randomly generated facts for two predicates where each fact invoked only trivial processing (incrementing a counter). We determined that the effects of combinatorial PRS algorithms could be avoided by batching new facts each time through its control loop. We modified the control loop to do so, and the performance improved remarkably. For a test case of 2,000 facts posted in 1 second, it reduced time to respond to the first (any) fact by 84%, reduced time to respond to all facts by 72%, and reduced memory usage by 83%. Experiments showed that a fact batch size near 55 was optimal for reducing response time, and any value between roughly 25 and 100 was near optimal.