Add Observations to Loop Optimizations Environment

[mostafaelhoushi]

• Adds inst2vec [Done], Autophase [Done], and Programl [In Progress] to the Loop Optimizations Environment
• to enable extracting ProGraml I created a small compute_programl.cc program. I tried first to place it in external/programl but was getting build errors, so I temporarily placed it in the loop opt example directory.

Issues:

• we need to first buld compute_autophase separately before we can run our example:

bazel build //compiler_gym/third_party/autophase:compute_autophase-prelinked

bazel run //examples/loop_optimizations_service:example

TODO (before merging):

• fix build scripts so that we won't need to build compute_autophase
• move compute_programl to a suitable directory
• complete Programl observation
• add unit test cases for each observation
• add AutoPhase dictionary observation
• verify build with CMake on Linux

[codecov-commenter]

Codecov Report

Merging #570 (7d42155) into development (0394e60) will increase coverage by 0.02%.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           development     #570      +/-   ##
===============================================
+ Coverage        88.17%   88.20%   +0.02%     
===============================================
  Files              114      114              
  Lines             6708     6708              
===============================================
+ Hits              5915     5917       +2     
+ Misses             793      791       -2     

Impacted Files	Coverage Δ
compiler_gym/service/proto/__init__.py	100.00% <ø> (ø)
...loop_tool/service/loop_tool_compilation_session.py	88.43% <0.00%> (-1.37%)	⬇️
compiler_gym/envs/compiler_env.py	91.83% <0.00%> (+0.49%)	⬆️
compiler_gym/views/observation.py	100.00% <0.00%> (+2.70%)	⬆️
compiler_gym/views/observation_space_spec.py	89.28% <0.00%> (+3.57%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0394e60...7d42155. Read the comment docs.

[mostafaelhoushi]

@ChrisCummins, I addressed the comments and enabled Programl observation (also moved compute_programl to compiler_gym/third_party/programl.

However, I am currently printing programl observation as a Json string. Is that how the originl CompilerGym LLVM environments print Programl observation?

[ChrisCummins]

@ChrisCummins, I addressed the comments and enabled Programl observation (also moved compute_programl to compiler_gym/third_party/programl.

Thanks! And yep, compiler_gym/third_party sounds like a good idea.

However, I am currently printing programl observation as a Json string. Is that how the originl CompilerGym LLVM environments print Programl observation?

This is exactly what the LLVM environment does:

CompilerGym/compiler_gym/envs/llvm/service/Observation.cc

Lines 101 to 111 in 11784d4

	// Serialize the graph to a JSON node link graph.
	json nodeLinkGraph;
	status = programl::graph::format::ProgramGraphToNodeLinkGraph(graph, &nodeLinkGraph);
	if (!status.ok()) {
	return Status(StatusCode::INTERNAL, status.error_message());
	}
	Opaque opaque;
	opaque.set_format(space == LlvmObservationSpace::PROGRAML ? "json://networkx/MultiDiGraph"
	: "json://");
	*opaque.mutable_data() = nodeLinkGraph.dump();
	reply.mutable_any_value()->PackFrom(opaque);

However, since merging #531, this could be made more efficient for the LLVM environment by embedding the ProgramGraph protocol buffer directly in the observation, removing the need to do a JSON serialize/deserialize. I haven't got around to doing that refactor yet though.

Cheers,
Chris

[ChrisCummins]

I agree with your TODO list in the PR message about adding some extra test cases to cover these new observations. Otherwise, LGTM!

[mostafaelhoushi]

Building on Linux is failing as I get this error (see CI log here):

-- Configuring done
CMake Error at build_tools/cmake/cg_macros.cmake:281 (add_dependencies):
  The dependency target
  "compiler_gym__third_party__autophase_compute_autophase" of target
  "examples__loop_optimizations_service__service_py__loops_opt_service" does
  not exist.
Call Stack (most recent call first):
  build_tools/cmake/cg_py_library.cmake:78 (cg_add_data_dependencies)
  examples/loop_optimizations_service/service_py/CMakeLists.txt:8 (cg_py_library)


CMake Error at build_tools/cmake/cg_macros.cmake:281 (add_dependencies):
  The dependency target
  "compiler_gym__third_party__programl_compute_programl" of target
  "examples__loop_optimizations_service__service_py__loops_opt_service" does
  not exist.
Call Stack (most recent call first):
  build_tools/cmake/cg_py_library.cmake:78 (cg_add_data_dependencies)
  examples/loop_optimizations_service/service_py/CMakeLists.txt:8 (cg_py_library)


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.
Error: Process completed with exit code 1.

I tried many times to fix it, and with the issue that CMake configuration takes multiple hours (see #595 ), it was really difficult to solve it. So I merged for now and will fix it later.

Cc @ChrisCummins @sogartar