Project Hephaestus for Compiler Testing

Hephaestus is a project for validating static typing procedures in compilers. As part of this project, we have developed numerous techniques to stress-test the implementation of type checkers in popular programming languages, such as Java, Kotlin, Groovy, and Scala. At the high level, our techniques generate type-intensive programs by either producing complex definitions from scratch or by combining synthetic code with application programming interfaces (APIs) from existing software libraries. To validate diverse aspects of static typing procedures (i.e., soundness, precision of type inference), we also enrich our approaches with fault-injection and semantics-preserving modes, along with their corresponding test oracles.

Tools: https://github.com/hephaestus-compiler-project

Results

Bugs found in compilers:

Summary: 262 (total) / 135 (fixed)

Well-typed programs (w/o type erasure) (report links)
Well-typed programs (w/ type erasure) (report links)
Ill-typed programs (w/o type erasure) (report links)
Ill-typed programs (w/ type erasure) (report links)

Team

Thodoris Sotiropoulos
Stefanos Chaliasos
Giorgos Drosos
Charalambos Mitropoulos
Benjamin Livshits
Diomidis Spinellis
Dimitris Mitropoulos
Zhendong Su

Publications

[1] API-driven Program Synthesis for Testing Static Typing Implementations. [artifact]
Thodoris Sotiropoulos, Stefanos Chaliasos, Zhendong Su.
In Proceedings of POPL 2024, London, UK, January 2024.

[2] Finding Typing Compiler Bugs. [artifact]
Stefanos Chaliasos*, Thodoris Sotiropoulos*, Diomidis Spinellis, Arthur Gervais, Benjamin Livshits, and Dimitris Mitropoulos.
In Proceedings of PLDI 2022, San Diego, US, June 2022.
* Both authors contributed equally to this work.


[3] Well-Typed Programs Can Go Wrong: A Study of Typing-Related Bugs in JVM Compilers. [artifact]
Stefanos Chaliasos*, Thodoris Sotiropoulos*, Georgios-Petros Drosos, Charalambos Mitropoulos, Dimitris Mitropoulos, and Diomidis Spinellis.
In Proceedings of OOPSLA 2021, Chicago, US, October 2021.
* Both authors contributed equally to this work.


last modified: 2023-11-09