Project Hephaestus for Compiler Testing
Hephaestus is a project for validating static typing procedures
in compilers. As part of this project,
we have developed numerous techniques to stress-test the
implementation of type checkers in popular programming languages,
such as Java, Kotlin, Groovy,
and Scala.
At the high level,
our techniques generate type-intensive programs by
either producing complex definitions from scratch or
by combining synthetic code with application programming interfaces (APIs)
from existing software libraries.
To validate diverse aspects of static typing procedures
(i.e., soundness, precision
of type inference),
we also enrich our approaches with fault-injection
and semantics-preserving
modes, along with their corresponding test oracles.
Tools: https://github.com/hephaestus-compiler-project
Results
Bugs found in compilers:
Summary: 262 (total) / 135 (fixed)
- Groovy bugs: 196 (total) / 121 (fixed)
- Kotlin bugs: 44 (total) / 11 (fixed)
- Java bugs: 11 (total) / 2 (fixed)
- Scala bugs: 11 (total) / 1 (fixed)
- Unexpected Compile Time Error bugs: 183 (total)
- Unexpected Runtime Behavior bugs: 29 (total)
- Crash bugs: 48 (total)
- Compilation Performance Issue bugs: 2 (total)
Well-typed programs (w/o type erasure) (report links)
Well-typed programs (w/ type erasure) (report links)
Ill-typed programs (w/o type erasure) (report links)
Ill-typed programs (w/ type erasure) (report links)
Team
Thodoris Sotiropoulos
Stefanos Chaliasos
Giorgos Drosos
Charalambos Mitropoulos
Benjamin Livshits
Diomidis Spinellis
Dimitris Mitropoulos
Zhendong Su
Publications
[1]
API-driven Program Synthesis for Testing Static Typing Implementations.
[artifact]
Thodoris Sotiropoulos, Stefanos Chaliasos, Zhendong Su.
In Proceedings of POPL 2024, London, UK, January 2024.
[2]
Finding Typing Compiler Bugs.
[artifact]
Stefanos Chaliasos*, Thodoris Sotiropoulos*, Diomidis Spinellis, Arthur Gervais, Benjamin Livshits, and Dimitris Mitropoulos.
In Proceedings of PLDI 2022, San Diego, US, June 2022.
*
Both authors contributed equally to this work.
[3]
Well-Typed Programs Can Go Wrong: A Study of Typing-Related Bugs in JVM Compilers.
[artifact]
Stefanos Chaliasos*, Thodoris Sotiropoulos*, Georgios-Petros Drosos, Charalambos Mitropoulos, Dimitris Mitropoulos, and Diomidis Spinellis.
In Proceedings of OOPSLA 2021, Chicago, US, October 2021.
*
Both authors contributed equally to this work.
last modified: 2023-11-09