Importing from JSON
In this example, we will use the same schema defined in the previous example to build an importer that re-constructs a register model from a JSON file.
We will use Python’s built-in JSON library
to read the input file, as well as the RDLImporter
helper class to help construct RDL components.
The full example code can be found in the systemrdl-compiler repository at:
examples/import_json.py
Walkthrough
Start by creating a JSON importer class extended from RDLImporter.
from systemrdl.importer import RDLImporter
class JSONImporter(RDLImporter):
Just like the JSON exporter example, we will define a function that converts each component type. This time, Python’s JSON library will read the input and present each component as a dictionary of keys:value pairs that represents its description.
Per-component conversion functions
When developing your importer, it is important to always validate inputs, and
create human-friendly error messages or warnings whenever appropriate.
Prior to decoding any component, we start by checking the validity of the JSON
dict object. Starting with the function to decode fields:
def decode_field(self, json_obj: dict) -> comp.Field:
# validate that this json object contains all the required fields
if 'inst_name' not in json_obj:
self.msg.fatal("JSON object is missing 'inst_name'", self.default_src_ref)
if 'lsb' not in json_obj:
self.msg.fatal("JSON object is missing 'lsb'", self.default_src_ref)
if 'msb' not in json_obj:
self.msg.fatal("JSON object is missing 'msb'", self.default_src_ref)
if 'reset' not in json_obj:
self.msg.fatal("JSON object is missing 'reset'", self.default_src_ref)
if 'sw_access' not in json_obj:
self.msg.fatal("JSON object is missing 'sw_access'", self.default_src_ref)
Next, we use some methods provided by the RDLImporter
to:
Create an anonymous definition of a field component
Assign some properties
Instantiate it
Note that the field instance is not fully instantiated yet. It still needs to be attached to a parent component. This will be done later outside of this function.
# Create an RDL field definition
comp_def = self.create_field_definition()
# Apply reset property if it was set
if json_obj['reset'] is not None:
self.assign_property(comp_def, 'reset', json_obj['reset'])
# decode and apply the sw access property.
# Since it uses the native enumeration name, we can do this directly
sw = AccessType[json_obj['sw_access']]
self.assign_property(comp_def, 'sw', sw)
# Instantiate the component definition
inst = self.instantiate_field(
comp_def,
json_obj['inst_name'],
json_obj['lsb'],
json_obj['msb'] - json_obj['lsb'] + 1
)
return inst
Next is the function that constructs the enclosing register component. The reg component is constructed in nearly the same way as before, except that here we iterate over all the child fields and attach them to the parent reg definition.
def decode_reg(self, json_obj: dict) -> comp.Reg:
# validate that this json object contains all the required fields
if 'inst_name' not in json_obj:
self.msg.fatal("JSON object is missing 'inst_name'", self.default_src_ref)
if 'addr_offset' not in json_obj:
self.msg.fatal("JSON object '%s' is missing 'addr_offset'" % json_obj['inst_name'], self.default_src_ref)
if 'children' not in json_obj:
self.msg.fatal("JSON object is missing 'children'", self.default_src_ref)
comp_def = self.create_reg_definition()
# Collect children
for child_json in json_obj['children']:
# Check that the child is the correct type. Reg can only contain fields
t = child_json.get('type', None)
if t != "field":
self.msg.fatal(
"Invalid child type '%s'" % t,
self.default_src_ref
)
# Convert each child component and add it to our reg definition
child_inst = self.decode_field(child_json)
self.add_child(comp_def, child_inst)
# Convert the definition into an instance
inst = self.instantiate_reg(
comp_def,
json_obj['inst_name'],
json_obj['addr_offset']
)
return inst
Decoding a regfile is nearly the same. Here, children can be multiple different types, so we call the appropriate decode function based on the ‘type’ key:
def decode_regfile(self, json_obj: dict) -> comp.Regfile:
if 'inst_name' not in json_obj:
self.msg.fatal("JSON object is missing 'inst_name'", self.default_src_ref)
if 'addr_offset' not in json_obj:
self.msg.fatal("JSON object '%s' is missing 'addr_offset'" % json_obj['inst_name'], self.default_src_ref)
if 'children' not in json_obj:
self.msg.fatal("JSON object is missing 'children'", self.default_src_ref)
comp_def = self.create_regfile_definition()
for child_json in json_obj['children']:
t = child_json.get('type', None)
if t == "regfile":
child_inst = self.decode_regfile(child_json)
elif t == "reg":
child_inst = self.decode_reg(child_json)
else:
self.msg.fatal(
"Invalid child type '%s'" % t,
self.default_src_ref
)
self.add_child(comp_def, child_inst)
inst = self.instantiate_regfile(
comp_def,
json_obj['inst_name'],
json_obj['addr_offset']
)
return inst
The addrmap decode function is nearly identical, however we need the option to produce a named definition instead of an anonymous component instantiation. This is because we want to be able to register the top-level addrmap in the root type namespace. The type namespace can only contain component definitions, not instances.
def decode_addrmap(self, json_obj: dict, is_top: bool=False) -> comp.Addrmap:
# validate that this json object contains all the required fields
if 'inst_name' not in json_obj:
self.msg.fatal("JSON object is missing 'inst_name'", self.default_src_ref)
if 'addr_offset' not in json_obj:
self.msg.fatal("JSON object '%s' is missing 'addr_offset'" % json_obj['inst_name'], self.default_src_ref)
if 'children' not in json_obj:
self.msg.fatal("JSON object is missing 'children'", self.default_src_ref)
if is_top:
# if this is the top node, then instantiation is skipped, and the
# definition inherits the inst name as its type name
comp_def = self.create_addrmap_definition(json_obj['inst_name'])
else:
# otherwise, create an anonymous definition
comp_def = self.create_addrmap_definition()
# Collect children
for child_json in json_obj['children']:
# Lookup the child type and call the appropriate conversion function
t = child_json.get('type', None)
if t == "addrmap":
child_inst = self.decode_addrmap(child_json)
elif t == "regfile":
child_inst = self.decode_regfile(child_json)
elif t == "reg":
child_inst = self.decode_reg(child_json)
else:
self.msg.fatal(
"Invalid child type '%s'" % t,
self.default_src_ref
)
# Add the child component to this
self.add_child(comp_def, child_inst)
if is_top:
# keep top-level addrmap as a definition. Skip instantiation
return comp_def
# For everything else, convert the definition into an instance
inst = self.instantiate_addrmap(
comp_def,
json_obj['inst_name'],
json_obj['addr_offset']
)
return inst
Loading JSON and decoding
Now that we have our decoding utility functions completed, we are ready to
implement the importer’s import_file()
entry function.
First, call the superclass. This ensures that per-component error message context works properly later.
def import_file(self, path: str) -> None:
super().import_file(path)
Next, load the JSON file. For this importer, we’ll be assuming that the top-level component is always an addrmap. Any time an assumption about the input is made, you should validate it to ensure it is true:
# Load the JSON from a file and convert it to primitive Python objects
with open(path, 'r', encoding='utf-8') as f:
json_obj = json.load(f)
# Make sure top level object is an addrmap type
if json_obj.get('type', None) != "addrmap":
self.msg.fatal(
"Top JSON object must be an addrmap type",
self.default_src_ref
)
Finally, call our decode function and register the resulting top-level component definition with the root namespace:
# Decode the JSON object
# Set is_top=True so that decode returns a definition rather than an instance
top_addrmap_def = self.decode_addrmap(json_obj, is_top=True)
# Register the top definition in the root namespace
self.register_root_component(top_addrmap_def)
Using our new importer
Importing data into the SystemRDL compiler is similar to compiling a file normally. You can even intermingle compiling RDL files with importing custom formats.
Create a
RDLCompilerinstanceBind any custom importers
Compile RDL inputs and/or import custom file formats
Run elaboration
Here we detect the file type based on its extension:
# Create a compiler session, and an importer attached to it
rdlc = RDLCompiler()
json_importer = JSONImporter(rdlc)
# import each JSON file provided from the command line
input_files = sys.argv[1:]
try:
for input_file in input_files:
# compile or import based on the file extension
ext = os.path.splitext(input_file)[1]
if ext == ".rdl":
rdlc.compile_file(input_file)
elif ext == ".json":
json_importer.import_file(input_file)
else:
rdlc.msg.fatal(
"Unknown file extension: %s" % ext,
FileSourceRef(input_file)
)
# Elaborate when done
root = rdlc.elaborate()
except RDLCompileError:
sys.exit(1)
Equivalence check
Let’s see if our importer is working properly. We can use the model printing listener from the first example to dump the register model:
from print_hierarchy import MyModelPrintingListener
walker = RDLWalker()
listener = MyModelPrintingListener()
walker.walk(root, listener)
Running the example code on tiny.rdl and its JSON counterpart we exported
in the previous example, tiny.json, we should see identical output:
$ ./import_json.py tiny.json
tiny
r1
[7:0] f1 sw=rw
[15:8] f2 sw=r
$ ./import_json.py tiny.rdl
tiny
r1
[7:0] f1 sw=rw
[15:8] f2 sw=r