Summary report generated on 2012-01-09T14:47:09Z (UTC) by Autobahn WebSockets v0.4.10.
Pass | Test case was executed and passed successfully. |
Non-Strict | Test case was executed and passed non-strictly. A non-strict behavior is one that does not adhere to a SHOULD-behavior as described in the protocol specification or a well-defined, canonical behavior that appears to be desirable but left open in the protocol specification. An implementation with non-strict behavior is still conformant to the protocol specification. |
Fail | Test case was executed and failed. An implementation which fails a test case - other than a performance/limits related one - is non-conforming to a MUST-behavior as described in the protocol specification. |
Info | Informational test case which detects certain implementation behavior left unspecified by the spec but nevertheless potentially interesting to implementors. |
Missing | Test case is missing, either because it was skipped via the test suite configuration or deactivated, i.e. because the implementation does not implement the tested feature or breaks during running the test case. |
1 Framing | AutobahnServer/0.4.10 | wslay | ||
1.1 Text Messages | ||||
Case 1.1.1 | Pass | 1000 | Pass | 1000 |
Case 1.1.2 | Pass | 1000 | Pass | 1000 |
Case 1.1.3 | Pass | 1000 | Pass | 1000 |
Case 1.1.4 | Pass | 1000 | Pass | 1000 |
Case 1.1.5 | Pass | 1000 | Pass | 1000 |
Case 1.1.6 | Pass | 1000 | Pass | 1000 |
Case 1.1.7 | Pass | 1000 | Pass | 1000 |
Case 1.1.8 | Pass | 1000 | Pass | 1000 |
1 Framing | AutobahnServer/0.4.10 | wslay | ||
1.2 Binary Messages | ||||
Case 1.2.1 | Pass | 1000 | Pass | 1000 |
Case 1.2.2 | Pass | 1000 | Pass | 1000 |
Case 1.2.3 | Pass | 1000 | Pass | 1000 |
Case 1.2.4 | Pass | 1000 | Pass | 1000 |
Case 1.2.5 | Pass | 1000 | Pass | 1000 |
Case 1.2.6 | Pass | 1000 | Pass | 1000 |
Case 1.2.7 | Pass | 1000 | Pass | 1000 |
Case 1.2.8 | Pass | 1000 | Pass | 1000 |
2 Pings/Pongs | AutobahnServer/0.4.10 | wslay | ||
Case 2.1 | Pass | 1000 | Pass | 1000 |
Case 2.2 | Pass | 1000 | Pass | 1000 |
Case 2.3 | Pass | 1000 | Pass | 1000 |
Case 2.4 | Pass | 1000 | Pass | 1000 |
Case 2.5 | Pass | 1002 | Pass | None |
Case 2.6 | Pass | 1000 | Pass | 1000 |
Case 2.7 | Pass | 1000 | Pass | 1000 |
Case 2.8 | Pass | 1000 | Pass | 1000 |
Case 2.9 | Pass | 1000 | Pass | 1000 |
Case 2.10 | Pass | 1000 | Pass | 1000 |
Case 2.11 | Pass | 1000 | Pass | 1000 |
3 Reserved Bits | AutobahnServer/0.4.10 | wslay | ||
Case 3.1 | Pass | 1002 | Pass | 1002 |
Case 3.2 | Pass | 1002 | Non-Strict | 1002 |
Case 3.3 | Pass | 1002 | Non-Strict | 1002 |
Case 3.4 | Pass | 1002 | Pass | 1002 |
Case 3.5 | Pass | 1002 | Pass | 1002 |
Case 3.6 | Pass | 1002 | Pass | 1002 |
Case 3.7 | Pass | 1002 | Pass | 1002 |
4 Opcodes | AutobahnServer/0.4.10 | wslay | ||
4.1 Non-control Opcodes | ||||
Case 4.1.1 | Pass | 1002 | Pass | 1002 |
Case 4.1.2 | Pass | 1002 | Pass | 1002 |
Case 4.1.3 | Pass | 1002 | Non-Strict | 1002 |
Case 4.1.4 | Pass | 1002 | Non-Strict | 1002 |
Case 4.1.5 | Pass | 1002 | Pass | 1002 |
4 Opcodes | AutobahnServer/0.4.10 | wslay | ||
4.2 Control Opcodes | ||||
Case 4.2.1 | Pass | 1002 | Pass | 1002 |
Case 4.2.2 | Pass | 1002 | Pass | 1002 |
Case 4.2.3 | Pass | 1002 | Non-Strict | 1002 |
Case 4.2.4 | Pass | 1002 | Non-Strict | 1002 |
Case 4.2.5 | Pass | 1002 | Pass | 1002 |
5 Fragmentation | AutobahnServer/0.4.10 | wslay | ||
Case 5.1 | Pass | 1002 | Pass | None |
Case 5.2 | Pass | 1002 | Pass | None |
Case 5.3 | Pass | 1000 | Pass | 1000 |
Case 5.4 | Pass | 1000 | Pass | 1000 |
Case 5.5 | Pass | 1000 | Pass | 1000 |
Case 5.6 | Pass | 1000 | Pass | 1000 |
Case 5.7 | Pass | 1000 | Pass | 1000 |
Case 5.8 | Pass | 1000 | Pass | 1000 |
Case 5.9 | Pass | 1002 | Pass | 1002 |
Case 5.10 | Pass | 1002 | Pass | 1002 |
Case 5.11 | Pass | 1002 | Pass | 1002 |
Case 5.12 | Pass | 1002 | Pass | 1002 |
Case 5.13 | Pass | 1002 | Pass | 1002 |
Case 5.14 | Pass | 1002 | Pass | 1002 |
Case 5.15 | Pass | 1002 | Non-Strict | 1002 |
Case 5.16 | Pass | 1002 | Pass | 1002 |
Case 5.17 | Pass | 1002 | Pass | 1002 |
Case 5.18 | Pass | 1002 | Pass | 1002 |
Case 5.19 | Pass | 1000 | Pass | 1000 |
Case 5.20 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.1 Valid UTF-8 with zero payload fragments | ||||
Case 6.1.1 | Pass | 1000 | Pass | 1000 |
Case 6.1.2 | Pass | 1000 | Pass | 1000 |
Case 6.1.3 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.2 Valid UTF-8 unfragmented, fragmented on code-points and within code-points | ||||
Case 6.2.1 | Pass | 1000 | Pass | 1000 |
Case 6.2.2 | Pass | 1000 | Pass | 1000 |
Case 6.2.3 | Pass | 1000 | Pass | 1000 |
Case 6.2.4 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.3 Invalid UTF-8 differently fragmented | ||||
Case 6.3.1 | Pass | 1007 | Pass | 1007 |
Case 6.3.2 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.4 Fail-fast on invalid UTF-8 | ||||
Case 6.4.1 | Pass | 1007 | Pass | 1007 |
Case 6.4.2 | Pass | 1007 | Pass | 1007 |
Case 6.4.3 | Pass | 1007 | Pass | 1007 |
Case 6.4.4 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.5 Some valid UTF-8 sequences | ||||
Case 6.5.1 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.6 All prefixes of a valid UTF-8 string that contains multi-byte code points | ||||
Case 6.6.1 | Pass | 1007 | Pass | 1007 |
Case 6.6.2 | Pass | 1000 | Pass | 1000 |
Case 6.6.3 | Pass | 1007 | Pass | 1007 |
Case 6.6.4 | Pass | 1007 | Pass | 1007 |
Case 6.6.5 | Pass | 1000 | Pass | 1000 |
Case 6.6.6 | Pass | 1007 | Pass | 1007 |
Case 6.6.7 | Pass | 1000 | Pass | 1000 |
Case 6.6.8 | Pass | 1007 | Pass | 1007 |
Case 6.6.9 | Pass | 1000 | Pass | 1000 |
Case 6.6.10 | Pass | 1007 | Pass | 1007 |
Case 6.6.11 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.7 First possible sequence of a certain length | ||||
Case 6.7.1 | Pass | 1000 | Pass | 1000 |
Case 6.7.2 | Pass | 1000 | Pass | 1000 |
Case 6.7.3 | Pass | 1000 | Pass | 1000 |
Case 6.7.4 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.8 First possible sequence length 5/6 (invalid codepoints) | ||||
Case 6.8.1 | Pass | 1007 | Pass | 1007 |
Case 6.8.2 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.9 Last possible sequence of a certain length | ||||
Case 6.9.1 | Pass | 1000 | Pass | 1000 |
Case 6.9.2 | Pass | 1000 | Pass | 1000 |
Case 6.9.3 | Pass | 1000 | Pass | 1000 |
Case 6.9.4 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.10 Last possible sequence length 4/5/6 (invalid codepoints) | ||||
Case 6.10.1 | Pass | 1007 | Pass | 1007 |
Case 6.10.2 | Pass | 1007 | Pass | 1007 |
Case 6.10.3 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.11 Other boundary conditions | ||||
Case 6.11.1 | Pass | 1000 | Pass | 1000 |
Case 6.11.2 | Pass | 1000 | Pass | 1000 |
Case 6.11.3 | Pass | 1000 | Pass | 1000 |
Case 6.11.4 | Pass | 1000 | Pass | 1000 |
Case 6.11.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.12 Unexpected continuation bytes | ||||
Case 6.12.1 | Pass | 1007 | Pass | 1007 |
Case 6.12.2 | Pass | 1007 | Pass | 1007 |
Case 6.12.3 | Pass | 1007 | Pass | 1007 |
Case 6.12.4 | Pass | 1007 | Pass | 1007 |
Case 6.12.5 | Pass | 1007 | Pass | 1007 |
Case 6.12.6 | Pass | 1007 | Pass | 1007 |
Case 6.12.7 | Pass | 1007 | Pass | 1007 |
Case 6.12.8 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.13 Lonely start characters | ||||
Case 6.13.1 | Pass | 1007 | Pass | 1007 |
Case 6.13.2 | Pass | 1007 | Pass | 1007 |
Case 6.13.3 | Pass | 1007 | Pass | 1007 |
Case 6.13.4 | Pass | 1007 | Pass | 1007 |
Case 6.13.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.14 Sequences with last continuation byte missing | ||||
Case 6.14.1 | Pass | 1007 | Pass | 1007 |
Case 6.14.2 | Pass | 1007 | Pass | 1007 |
Case 6.14.3 | Pass | 1007 | Pass | 1007 |
Case 6.14.4 | Pass | 1007 | Pass | 1007 |
Case 6.14.5 | Pass | 1007 | Pass | 1007 |
Case 6.14.6 | Pass | 1007 | Pass | 1007 |
Case 6.14.7 | Pass | 1007 | Pass | 1007 |
Case 6.14.8 | Pass | 1007 | Pass | 1007 |
Case 6.14.9 | Pass | 1007 | Pass | 1007 |
Case 6.14.10 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.15 Concatenation of incomplete sequences | ||||
Case 6.15.1 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.16 Impossible bytes | ||||
Case 6.16.1 | Pass | 1007 | Pass | 1007 |
Case 6.16.2 | Pass | 1007 | Pass | 1007 |
Case 6.16.3 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.17 Examples of an overlong ASCII character | ||||
Case 6.17.1 | Pass | 1007 | Pass | 1007 |
Case 6.17.2 | Pass | 1007 | Pass | 1007 |
Case 6.17.3 | Pass | 1007 | Pass | 1007 |
Case 6.17.4 | Pass | 1007 | Pass | 1007 |
Case 6.17.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.18 Maximum overlong sequences | ||||
Case 6.18.1 | Pass | 1007 | Pass | 1007 |
Case 6.18.2 | Pass | 1007 | Pass | 1007 |
Case 6.18.3 | Pass | 1007 | Pass | 1007 |
Case 6.18.4 | Pass | 1007 | Pass | 1007 |
Case 6.18.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.19 Overlong representation of the NUL character | ||||
Case 6.19.1 | Pass | 1007 | Pass | 1007 |
Case 6.19.2 | Pass | 1007 | Pass | 1007 |
Case 6.19.3 | Pass | 1007 | Pass | 1007 |
Case 6.19.4 | Pass | 1007 | Pass | 1007 |
Case 6.19.5 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.20 Single UTF-16 surrogates | ||||
Case 6.20.1 | Pass | 1007 | Pass | 1007 |
Case 6.20.2 | Pass | 1007 | Pass | 1007 |
Case 6.20.3 | Pass | 1007 | Pass | 1007 |
Case 6.20.4 | Pass | 1007 | Pass | 1007 |
Case 6.20.5 | Pass | 1007 | Pass | 1007 |
Case 6.20.6 | Pass | 1007 | Pass | 1007 |
Case 6.20.7 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.21 Paired UTF-16 surrogates | ||||
Case 6.21.1 | Pass | 1007 | Pass | 1007 |
Case 6.21.2 | Pass | 1007 | Pass | 1007 |
Case 6.21.3 | Pass | 1007 | Pass | 1007 |
Case 6.21.4 | Pass | 1007 | Pass | 1007 |
Case 6.21.5 | Pass | 1007 | Pass | 1007 |
Case 6.21.6 | Pass | 1007 | Pass | 1007 |
Case 6.21.7 | Pass | 1007 | Pass | 1007 |
Case 6.21.8 | Pass | 1007 | Pass | 1007 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.22 Non-character code points (valid UTF-8) | ||||
Case 6.22.1 | Pass | 1000 | Pass | 1000 |
Case 6.22.2 | Pass | 1000 | Pass | 1000 |
Case 6.22.3 | Pass | 1000 | Pass | 1000 |
Case 6.22.4 | Pass | 1000 | Pass | 1000 |
Case 6.22.5 | Pass | 1000 | Pass | 1000 |
Case 6.22.6 | Pass | 1000 | Pass | 1000 |
Case 6.22.7 | Pass | 1000 | Pass | 1000 |
Case 6.22.8 | Pass | 1000 | Pass | 1000 |
Case 6.22.9 | Pass | 1000 | Pass | 1000 |
Case 6.22.10 | Pass | 1000 | Pass | 1000 |
Case 6.22.11 | Pass | 1000 | Pass | 1000 |
Case 6.22.12 | Pass | 1000 | Pass | 1000 |
Case 6.22.13 | Pass | 1000 | Pass | 1000 |
Case 6.22.14 | Pass | 1000 | Pass | 1000 |
Case 6.22.15 | Pass | 1000 | Pass | 1000 |
Case 6.22.16 | Pass | 1000 | Pass | 1000 |
Case 6.22.17 | Pass | 1000 | Pass | 1000 |
Case 6.22.18 | Pass | 1000 | Pass | 1000 |
Case 6.22.19 | Pass | 1000 | Pass | 1000 |
Case 6.22.20 | Pass | 1000 | Pass | 1000 |
Case 6.22.21 | Pass | 1000 | Pass | 1000 |
Case 6.22.22 | Pass | 1000 | Pass | 1000 |
Case 6.22.23 | Pass | 1000 | Pass | 1000 |
Case 6.22.24 | Pass | 1000 | Pass | 1000 |
Case 6.22.25 | Pass | 1000 | Pass | 1000 |
Case 6.22.26 | Pass | 1000 | Pass | 1000 |
Case 6.22.27 | Pass | 1000 | Pass | 1000 |
Case 6.22.28 | Pass | 1000 | Pass | 1000 |
Case 6.22.29 | Pass | 1000 | Pass | 1000 |
Case 6.22.30 | Pass | 1000 | Pass | 1000 |
Case 6.22.31 | Pass | 1000 | Pass | 1000 |
Case 6.22.32 | Pass | 1000 | Pass | 1000 |
Case 6.22.33 | Pass | 1000 | Pass | 1000 |
Case 6.22.34 | Pass | 1000 | Pass | 1000 |
6 UTF-8 Handling | AutobahnServer/0.4.10 | wslay | ||
6.23 Unicode replacement character | ||||
Case 6.23.1 | Pass | 1000 | Pass | 1000 |
7 Close Handling | AutobahnServer/0.4.10 | wslay | ||
7.1 Basic close behavior (fuzzer initiated) | ||||
Case 7.1.1 | Pass | 1000 | Pass | 1000 |
Case 7.1.2 | Pass | 1000 | Pass | 1000 |
Case 7.1.3 | Pass | 1000 | Pass | 1000 |
Case 7.1.4 | Pass | 1000 | Pass | 1000 |
Case 7.1.5 | Pass | 1000 | Pass | 1000 |
Case 7.1.6 | Info | 1000 | Info | 1000 |
7 Close Handling | AutobahnServer/0.4.10 | wslay | ||
7.3 Close frame structure: payload length (fuzzer initiated) | ||||
Case 7.3.1 | Pass | 1000 | Pass | None |
Case 7.3.2 | Pass | 1002 | Pass | None |
Case 7.3.3 | Pass | 1000 | Pass | 1000 |
Case 7.3.4 | Pass | 1000 | Pass | 1000 |
Case 7.3.5 | Pass | 1000 | Pass | 1000 |
Case 7.3.6 | Pass | 1002 | Pass | None |
7 Close Handling | AutobahnServer/0.4.10 | wslay | ||
7.5 Close frame structure: payload value (fuzzer initiated) | ||||
Case 7.5.1 | Pass | 1007 | Pass | 1007 |
7 Close Handling | AutobahnServer/0.4.10 | wslay | ||
7.7 Close frame structure: valid close codes (fuzzer initiated) | ||||
Case 7.7.1 | Pass | 1000 | Pass | 1000 |
Case 7.7.2 | Pass | 1000 | Pass | 1001 |
Case 7.7.3 | Pass | 1000 | Pass | 1002 |
Case 7.7.4 | Pass | 1000 | Pass | 1003 |
Case 7.7.5 | Pass | 1000 | Pass | 1007 |
Case 7.7.6 | Pass | 1000 | Pass | 1008 |
Case 7.7.7 | Pass | 1000 | Pass | 1009 |
Case 7.7.8 | Pass | 1000 | Pass | 1010 |
Case 7.7.9 | Pass | 1000 | Pass | 1011 |
Case 7.7.10 | Pass | 1000 | Pass | 3000 |
Case 7.7.11 | Pass | 1000 | Pass | 3999 |
Case 7.7.12 | Pass | 1000 | Pass | 4000 |
Case 7.7.13 | Pass | 1000 | Pass | 4999 |
7 Close Handling | AutobahnServer/0.4.10 | wslay | ||
7.9 Close frame structure: invalid close codes (fuzzer initiated) | ||||
Case 7.9.1 | Pass | 1002 | Pass | 1002 |
Case 7.9.2 | Pass | 1002 | Pass | 1002 |
Case 7.9.3 | Pass | 1002 | Pass | 1002 |
Case 7.9.4 | Pass | 1002 | Pass | 1002 |
Case 7.9.5 | Pass | 1002 | Pass | 1002 |
Case 7.9.6 | Pass | 1002 | Pass | 1002 |
Case 7.9.7 | Pass | 1002 | Pass | 1002 |
Case 7.9.8 | Pass | 1002 | Pass | 1002 |
Case 7.9.9 | Pass | 1002 | Pass | 1002 |
Case 7.9.10 | Pass | 1002 | Pass | 1002 |
Case 7.9.11 | Pass | 1002 | Pass | 1002 |
Case 7.9.12 | Pass | 1002 | Pass | 1002 |
Case 7.9.13 | Pass | 1002 | Pass | 1002 |
7 Close Handling | AutobahnServer/0.4.10 | wslay | ||
7.13 Informational close information (fuzzer initiated) | ||||
Case 7.13.1 | Info | 1002 | Info | 1002 |
Case 7.13.2 | Info | 1002 | Info | 1002 |
9 Limits/Performance | AutobahnServer/0.4.10 | wslay | ||
9.1 Text Message (increasing size) | ||||
Case 9.1.1 | Pass 90 ms | 1000 | Pass 40 ms | 1000 |
Case 9.1.2 | Pass 331 ms | 1000 | Pass 154 ms | 1000 |
Case 9.1.3 | Pass 1318 ms | 1000 | Pass 621 ms | 1000 |
Case 9.1.4 | Pass 5285 ms | 1000 | Pass 2469 ms | 1000 |
Case 9.1.5 | Pass 10538 ms | 1000 | Pass 4917 ms | 1000 |
Case 9.1.6 | Pass 21119 ms | 1000 | Pass 9867 ms | 1000 |
9 Limits/Performance | AutobahnServer/0.4.10 | wslay | ||
9.2 Binary Message (increasing size) | ||||
Case 9.2.1 | Pass 30 ms | 1000 | Pass 12 ms | 1000 |
Case 9.2.2 | Pass 120 ms | 1000 | Pass 46 ms | 1000 |
Case 9.2.3 | Pass 477 ms | 1000 | Pass 191 ms | 1000 |
Case 9.2.4 | Pass 1902 ms | 1000 | Pass 764 ms | 1000 |
Case 9.2.5 | Pass 3789 ms | 1000 | Pass 1472 ms | 1000 |
Case 9.2.6 | Pass 7638 ms | 1000 | Pass 3027 ms | 1000 |
9 Limits/Performance | AutobahnServer/0.4.10 | wslay | ||
9.3 Fragmented Text Message (fixed size, increasing fragment size) | ||||
Case 9.3.1 | Pass 8014 ms | 1000 | Pass 3453 ms | 1000 |
Case 9.3.2 | Pass 5916 ms | 1000 | Pass 2716 ms | 1000 |
Case 9.3.3 | Pass 5426 ms | 1000 | Pass 2517 ms | 1000 |
Case 9.3.4 | Pass 5277 ms | 1000 | Pass 2428 ms | 1000 |
Case 9.3.5 | Pass 5224 ms | 1000 | Pass 2415 ms | 1000 |
Case 9.3.6 | Pass 5242 ms | 1000 | Pass 2409 ms | 1000 |
Case 9.3.7 | Pass 5234 ms | 1000 | Pass 2418 ms | 1000 |
Case 9.3.8 | Pass 5216 ms | 1000 | Pass 2428 ms | 1000 |
Case 9.3.9 | Pass 5221 ms | 1000 | Pass 2422 ms | 1000 |
9 Limits/Performance | AutobahnServer/0.4.10 | wslay | ||
9.4 Fragmented Binary Message (fixed size, increasing fragment size) | ||||
Case 9.4.1 | Pass 4550 ms | 1000 | Pass 1717 ms | 1000 |
Case 9.4.2 | Pass 2563 ms | 1000 | Pass 993 ms | 1000 |
Case 9.4.3 | Pass 2022 ms | 1000 | Pass 773 ms | 1000 |
Case 9.4.4 | Pass 1883 ms | 1000 | Pass 708 ms | 1000 |
Case 9.4.5 | Pass 1829 ms | 1000 | Pass 703 ms | 1000 |
Case 9.4.6 | Pass 1822 ms | 1000 | Pass 694 ms | 1000 |
Case 9.4.7 | Pass 1827 ms | 1000 | Pass 699 ms | 1000 |
Case 9.4.8 | Pass 1834 ms | 1000 | Pass 684 ms | 1000 |
Case 9.4.9 | Pass 1798 ms | 1000 | Pass 686 ms | 1000 |
9 Limits/Performance | AutobahnServer/0.4.10 | wslay | ||
9.5 Text Message (fixed size, increasing chop size) | ||||
Case 9.5.1 | Pass 1541 ms | 1000 | Pass 1597 ms | 1000 |
Case 9.5.2 | Pass 1322 ms | 1000 | Pass 1098 ms | 1000 |
Case 9.5.3 | Pass 1313 ms | 1000 | Pass 857 ms | 1000 |
Case 9.5.4 | Pass 1313 ms | 1000 | Pass 740 ms | 1000 |
Case 9.5.5 | Pass 1313 ms | 1000 | Pass 668 ms | 1000 |
Case 9.5.6 | Pass 1316 ms | 1000 | Pass 636 ms | 1000 |
9 Limits/Performance | AutobahnServer/0.4.10 | wslay | ||
9.6 Binary Text Message (fixed size, increasing chop size) | ||||
Case 9.6.1 | Pass 1194 ms | 1000 | Pass 1160 ms | 1000 |
Case 9.6.2 | Pass 673 ms | 1000 | Pass 676 ms | 1000 |
Case 9.6.3 | Pass 486 ms | 1000 | Pass 432 ms | 1000 |
Case 9.6.4 | Pass 481 ms | 1000 | Pass 310 ms | 1000 |
Case 9.6.5 | Pass 478 ms | 1000 | Pass 258 ms | 1000 |
Case 9.6.6 | Pass 473 ms | 1000 | Pass 219 ms | 1000 |
9 Limits/Performance | AutobahnServer/0.4.10 | wslay | ||
9.7 Text Message Roundtrip Time (fixed number, increasing size) | ||||
Case 9.7.1 | Pass 246 ms | 1000 | Pass 115 ms | 1000 |
Case 9.7.2 | Pass 312 ms | 1000 | Pass 143 ms | 1000 |
Case 9.7.3 | Pass 408 ms | 1000 | Pass 169 ms | 1000 |
Case 9.7.4 | Pass 788 ms | 1000 | Pass 291 ms | 1000 |
Case 9.7.5 | Pass 2272 ms | 1000 | Pass 755 ms | 1000 |
Case 9.7.6 | Pass 8201 ms | 1000 | Pass 2644 ms | 1000 |
9 Limits/Performance | AutobahnServer/0.4.10 | wslay | ||
9.8 Binary Message Roundtrip Time (fixed number, increasing size) | ||||
Case 9.8.1 | Pass 256 ms | 1000 | Pass 118 ms | 1000 |
Case 9.8.2 | Pass 280 ms | 1000 | Pass 127 ms | 1000 |
Case 9.8.3 | Pass 315 ms | 1000 | Pass 140 ms | 1000 |
Case 9.8.4 | Pass 478 ms | 1000 | Pass 186 ms | 1000 |
Case 9.8.5 | Pass 1025 ms | 1000 | Pass 343 ms | 1000 |
Case 9.8.6 | Pass 3230 ms | 1000 | Pass 971 ms | 1000 |
10 Autobahn Protocol Options | AutobahnServer/0.4.10 | wslay | ||
10.1 Auto-Fragmentation | ||||
Case 10.1.1 | Pass | 1000 | Pass | 1000 |
Case Description
Send text message with payload 0.
Case Expectation
Receive echo'ed text message (with empty payload). Clean close with normal code.
Case Description
Send text message message with payload of length 125.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 126.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 127.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 128.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 65535.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 65536.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send text message message with payload of length 65536. Sent out data in chops of 997 octets.
Case Expectation
Receive echo'ed text message (with payload as sent). Clean close with normal code.
Case Description
Send binary message with payload 0.
Case Expectation
Receive echo'ed binary message (with empty payload). Clean close with normal code.
Case Description
Send binary message message with payload of length 125.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 126.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 127.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 128.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 65535.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 65536.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send binary message message with payload of length 65536. Sent out data in chops of 997 octets.
Case Expectation
Receive echo'ed binary message (with payload as sent). Clean close with normal code.
Case Description
Send ping without payload.
Case Expectation
Pong (with empty payload) is sent in reply to Ping. Clean close with normal code.
Case Description
Send ping with small text payload.
Case Expectation
Pong with payload echo'ed is sent in reply to Ping. Clean close with normal code.
Case Description
Send ping with small binary (non UTF-8) payload.
Case Expectation
Pong with payload echo'ed is sent in reply to Ping. Clean close with normal code.
Case Description
Send ping with binary payload of 125 octets.
Case Expectation
Pong with payload echo'ed is sent in reply to Ping. Clean close with normal code.
Case Description
Send ping with binary payload of 126 octets.
Case Expectation
Connection is failed immediately (1002/Protocol Error), since control frames are only allowed to have payload up to and including 125 octets..
Case Description
Send ping with binary payload of 125 octets, send in octet-wise chops.
Case Expectation
Pong with payload echo'ed is sent in reply to Ping. Implementations must be TCP clean. Clean close with normal code.
Case Description
Send unsolicited pong without payload. Verify nothing is received. Clean close with normal code.
Case Expectation
Nothing.
Case Description
Send unsolicited pong with payload. Verify nothing is received. Clean close with normal code.
Case Expectation
Nothing.
Case Description
Send unsolicited pong with payload. Send ping with payload. Verify pong for ping is received.
Case Expectation
Nothing in reply to own Pong, but Pong with payload echo'ed in reply to Ping. Clean close with normal code.
Case Description
Send 10 Pings with payload.
Case Expectation
Pongs for our Pings with all the payloads. Note: This is not required by the Spec .. but we check for this behaviour anyway. Clean close with normal code.
Case Description
Send 10 Pings with payload. Send out octets in octet-wise chops.
Case Expectation
Pongs for our Pings with all the payloads. Note: This is not required by the Spec .. but we check for this behaviour anyway. Clean close with normal code.
Case Description
Send small text message with RSV = 1.
Case Expectation
The connection is failed immediately (1002/protocol error), since RSV must be 0, when no extension defining RSV meaning has been negoiated.
Case Description
Send small text message, then send again with RSV = 2, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since RSV must be 0, when no extension defining RSV meaning has been negoiated. The Pong is not received.
Case Description
Send small text message, then send again with RSV = 3, then send Ping. Octets are sent in frame-wise chops. Octets are sent in octet-wise chops.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since RSV must be 0, when no extension defining RSV meaning has been negoiated. The Pong is not received.
Case Description
Send small text message, then send again with RSV = 4, then send Ping. Octets are sent in octet-wise chops.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since RSV must be 0, when no extension defining RSV meaning has been negoiated. The Pong is not received.
Case Description
Send small binary message with RSV = 5.
Case Expectation
The connection is failed immediately, since RSV must be 0.
Case Description
Send Ping with RSV = 6.
Case Expectation
The connection is failed immediately, since RSV must be 0.
Case Description
Send Close with RSV = 7.
Case Expectation
The connection is failed immediately, since RSV must be 0.
Case Description
Send frame with reserved non-control Opcode = 3.
Case Expectation
The connection is failed immediately.
Case Description
Send frame with reserved non-control Opcode = 4 and non-empty payload.
Case Expectation
The connection is failed immediately.
Case Description
Send small text message, then send frame with reserved non-control Opcode = 5, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send small text message, then send frame with reserved non-control Opcode = 6 and non-empty payload, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send small text message, then send frame with reserved non-control Opcode = 7 and non-empty payload, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send frame with reserved control Opcode = 11.
Case Expectation
The connection is failed immediately.
Case Description
Send frame with reserved control Opcode = 12 and non-empty payload.
Case Expectation
The connection is failed immediately.
Case Description
Send small text message, then send frame with reserved control Opcode = 13, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send small text message, then send frame with reserved control Opcode = 14 and non-empty payload, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send small text message, then send frame with reserved control Opcode = 15 and non-empty payload, then send Ping.
Case Expectation
Echo for first message is received, but then connection is failed immediately, since reserved opcode frame is used. A Pong is not received.
Case Description
Send Ping fragmented into 2 fragments.
Case Expectation
Connection is failed immediately, since control message MUST NOT be fragmented.
Case Description
Send Pong fragmented into 2 fragments.
Case Expectation
Connection is failed immediately, since control message MUST NOT be fragmented.
Case Description
Send text Message fragmented into 2 fragments.
Case Expectation
Message is processed and echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, octets are sent in frame-wise chops.
Case Expectation
Message is processed and echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, octets are sent in octet-wise chops.
Case Expectation
Message is processed and echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, one ping with payload in-between.
Case Expectation
A pong is received, then the message is echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, one ping with payload in-between. Octets are sent in frame-wise chops.
Case Expectation
A pong is received, then the message is echo'ed back to us.
Case Description
Send text Message fragmented into 2 fragments, one ping with payload in-between. Octets are sent in octet-wise chops.
Case Expectation
A pong is received, then the message is echo'ed back to us.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = true, where there is nothing to continue, sent in one chop.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = true, where there is nothing to continue, sent in per-frame chops.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = true, where there is nothing to continue, sent in octet-wise chops.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = false, where there is nothing to continue, sent in one chop.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = false, where there is nothing to continue, sent in per-frame chops.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send unfragmented Text Message after Continuation Frame with FIN = false, where there is nothing to continue, sent in octet-wise chops.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send text Message fragmented into 2 fragments, then Continuation Frame with FIN = false where there is nothing to continue, then unfragmented Text Message, all sent in one chop.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Repeated 2x: Continuation Frame with FIN = false (where there is nothing to continue), then text Message fragmented into 2 fragments.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Repeated 2x: Continuation Frame with FIN = true (where there is nothing to continue), then text Message fragmented into 2 fragments.
Case Expectation
The connection is failed immediately, since there is no message to continue.
Case Description
Send text Message fragmented into 2 fragments, with both frame opcodes set to text, sent in one chop.
Case Expectation
The connection is failed immediately, since all data frames after the initial data frame must have opcode 0.
Case Description
A fragmented text message is sent in multiple frames. After
sending the first 2 frames of the text message, a Ping is sent. Then we wait 1s,
then we send 2 more text fragments, another Ping and then the final text fragment.
Everything is legal.
Case Expectation
The peer immediately answers the first Ping before
it has received the last text message fragment. The peer pong's back the Ping's
payload exactly, and echo's the payload of the fragmented message back to us.
Case Description
Same as Case 5.19, but send all frames with SYNC = True.
Note, this does not change the octets sent in any way, only how the stream
is chopped up on the wire.
Case Expectation
Same as Case 5.19. Implementations must be agnostic to how
octet stream is chopped up on wire (must be TCP clean).
Case Description
Send text message of length 0.
Case Expectation
A message is echo'ed back to us (with empty payload).
Case Description
Send fragmented text message, 3 fragments each of length 0.
Case Expectation
A message is echo'ed back to us (with empty payload).
Case Description
Send fragmented text message, 3 fragments, first and last of length 0, middle non-empty.
Case Expectation
A message is echo'ed back to us (with payload = payload of middle fragment).
Case Description
Send a valid UTF-8 text message in one fragment.
MESSAGE:
Hello-µ@ßöäüà á-UTF-8!!
48656c6c6f2dc2b540c39fc3b6c3a4c3bcc3a0c3a12d5554462d382121
Case Expectation
The message is echo'ed back to us.
Case Description
Send a valid UTF-8 text message in two fragments, fragmented on UTF-8 code point boundary.
MESSAGE FRAGMENT 1:
Hello-µ@ßöä
48656c6c6f2dc2b540c39fc3b6c3a4
MESSAGE FRAGMENT 2:
üà á-UTF-8!!
c3bcc3a0c3a12d5554462d382121
Case Expectation
The message is echo'ed back to us.
Case Description
Send a valid UTF-8 text message in fragments of 1 octet, resulting in frames ending on positions which are not code point ends.
MESSAGE:
Hello-µ@ßöäüà á-UTF-8!!
48656c6c6f2dc2b540c39fc3b6c3a4c3bcc3a0c3a12d5554462d382121
Case Expectation
The message is echo'ed back to us.
Case Description
Send a valid UTF-8 text message in fragments of 1 octet, resulting in frames ending on positions which are not code point ends.
MESSAGE:
κόσμε
cebae1bdb9cf83cebcceb5
Case Expectation
The message is echo'ed back to us.
Case Description
Send invalid UTF-8 text message unfragmented.
MESSAGE:
Îºá½¹ÏƒÎ¼Îµí €edited
cebae1bdb9cf83cebcceb5eda080656469746564
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send invalid UTF-8 text message in fragments of 1 octet, resulting in frames ending on positions which are not code point ends.
MESSAGE:
Îºá½¹ÏƒÎ¼Îµí €edited
cebae1bdb9cf83cebcceb5eda080656469746564
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send invalid UTF-8 text message in 3 fragments (frames).
First frame payload is valid, then wait, then 2nd frame which contains the payload making the sequence invalid, then wait, then 3rd frame with rest.
Note that PART1 and PART3 are valid UTF-8 in themselves, PART2 is a 0x11000 encoded as in the UTF-8 integer encoding scheme, but the codepoint is invalid (out of range).
MESSAGE PARTS:
PART1 = κόσμε (cebae1bdb9cf83cebcceb5)
PART2 = ô€€ (f4908080)
PART3 = edited (656469746564)
Case Expectation
The first frame is accepted, we expect to timeout on the first wait. The 2nd frame should be rejected immediately (fail fast on UTF-8). If we timeout, we expect the connection is failed at least then, since the complete message payload is not valid UTF-8.
Case Description
Same as Case 6.4.1, but in 2nd frame, we send only up to and including the octet making the complete payload invalid.
MESSAGE PARTS:
PART1 = κόσμεô (cebae1bdb9cf83cebcceb5f4)
PART2 = (90)
PART3 = €€edited (8080656469746564)
Case Expectation
The first frame is accepted, we expect to timeout on the first wait. The 2nd frame should be rejected immediately (fail fast on UTF-8). If we timeout, we expect the connection is failed at least then, since the complete message payload is not valid UTF-8.
Case Description
Same as Case 6.4.1, but we send message not in 3 frames, but in 3 chops of the same message frame.
MESSAGE PARTS:
PART1 = κόσμε (cebae1bdb9cf83cebcceb5)
PART2 = ô€€ (f4908080)
PART3 = edited (656469746564)
Case Expectation
The first chop is accepted, we expect to timeout on the first wait. The 2nd chop should be rejected immediately (fail fast on UTF-8). If we timeout, we expect the connection is failed at least then, since the complete message payload is not valid UTF-8.
Case Description
Same as Case 6.4.2, but we send message not in 3 frames, but in 3 chops of the same message frame.
MESSAGE PARTS:
PART1 = κόσμεô (cebae1bdb9cf83cebcceb5f4)
PART2 = (90)
PART3 = ()
Case Expectation
The first chop is accepted, we expect to timeout on the first wait. The 2nd chop should be rejected immediately (fail fast on UTF-8). If we timeout, we expect the connection is failed at least then, since the complete message payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κόσμε
cebae1bdb9cf83cebcceb5
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
Î
ce
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κ
ceba
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
뼇
cebae1
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
κá½
cebae1bd
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κό
cebae1bdb9
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
κόÏ
cebae1bdb9cf
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κόσ
cebae1bdb9cf83
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
κόσÎ
cebae1bdb9cf83ce
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κόσμ
cebae1bdb9cf83cebc
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is not valid UTF-8 in one fragment.
MESSAGE:
κόσμÎ
cebae1bdb9cf83cebcce
Case Expectation
The connection is failed immediately, since the payload is not valid UTF-8.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE:
κόσμε
cebae1bdb9cf83cebcceb5
Case Expectation
The message is echo'ed back to us.
Case Description
Send a text message with payload which is valid UTF-8 in one fragment.
MESSAGE: