Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[entropy_src/dv] Find out what happened to the group coverage #20565

Closed
h-filali opened this issue Dec 5, 2023 · 3 comments · Fixed by #20616
Closed

[entropy_src/dv] Find out what happened to the group coverage #20565

h-filali opened this issue Dec 5, 2023 · 3 comments · Fixed by #20616
Assignees
Labels
Component:DV DV issue: testbench, test case, etc. IP:entropy_src Type:Bug Bugs

Comments

@h-filali
Copy link

h-filali commented Dec 5, 2023

Description

Between the 16. and the 19. of October something caused the entropy source coverage to plummet to around 30 to 40%.
To verify this I ran the following command with a 0.1 reseed-multiplier:
./util/dvsim/dvsim.py hw/ip/entropy_src/dv/entropy_src_sim_cfg.hjson -i all -t vcs --reseed-multiplier 0.1 --cov
image

We need to find out what happened and fix this issue.

@h-filali h-filali self-assigned this Dec 5, 2023
@h-filali h-filali added Component:DV DV issue: testbench, test case, etc. Type:Bug Bugs IP:entropy_src labels Dec 5, 2023
@h-filali
Copy link
Author

h-filali commented Dec 6, 2023

image
These are the results after a full regression using xcelium with the following command:
./util/dvsim/dvsim.py hw/ip/entropy_src/dv/entropy_src_sim_cfg.hjson -i all -t xcelium --cov
Having a closer look at the coverage groups looks like this:
image

@h-filali
Copy link
Author

h-filali commented Dec 7, 2023

Status update:
I was able to narrow the issue down to PR #20036
There now seems to be an issue with the random variables of the cfg class. The rand variables don't seem to be random at all.
They all seem to have the same value, no matter what seed is used.
This likely also affects other IP blocks. In the dashboard the group coverage goes down for several blocks between the nightly regression before and after the above PR.

@h-filali
Copy link
Author

h-filali commented Dec 8, 2023

Status update #2:
The PR mentioned in the previous comment changes seed lengths (e.g. for the simulation and OTP) from 32 to 256 bits and systemverilog can only deal with 32 bit seeds. For this reason xrun just resorts to using a default seed, which means that we weren't really doing directed random tests in the past 1-2 months. This also explains why our coverage went down.

Furthermore, since the seed that is generated for systemverilog is not only used for seeding the systemverilog random variables, we can't just change this seed to 32 bits. A potential solution is to take the lowest 32 bits of the 256 bit seed and use it for the command that is issued when the test is being deployed. If I understand correctly, this happens in Deploy.py. There is a dict called mandatory_cmd_attrs where the data for the deploy command comes from. I'll continue working on this on monday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component:DV DV issue: testbench, test case, etc. IP:entropy_src Type:Bug Bugs
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant