Skip to content

feat: update tiled policy #300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ZohebShaikh wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: update tiled policy #300

ZohebShaikh wants to merge 1 commit into from

Conversation

@ZohebShaikh
Copy link
Collaborator

@ZohebShaikh ZohebShaikh commented Feb 2, 2026 •
edited
Loading

I have done benchmarking using benchstat.

The commands used are

# Benchmark with old policy
opa bench -b bundle.tar.gz -d ./policy/ "data.diamond.policy.tiled.user_sessions" --format gobench --count 10| tee ./old-181.txt
# Benchmark with new policy
opa bench -b bundle.tar.gz -d ./policy/ "data.diamond.policy.tiled.user_sessions" --format gobench --count 10| tee ./new-181.txt

benchstat --format csv ./new-181.txt ./old-181.txt > benchstat-output-181-csv.txt

There are 2 cases I have benchmarked

  1. When a super_admin request user_session they will get [*] vs all({"proposal","beamline","visit"}) so this will be definetly faster and the benchmarks prove that

New: 400 microseconds vs
Old: 500 milliseconds

  1. The interesting stats is for a user with 181 user-session and not a admin (ie will receive all the sessions)
    New : 360 microseconds vs
    Old: 52 seconds

I have attached the benchstat csv output below.

@ZohebShaikh ZohebShaikh mentioned this pull request Feb 2, 2026
Open
@ZohebShaikh
Copy link
Collaborator Author

ZohebShaikh commented Feb 2, 2026

  1. Super admin benchmark
FIELD1 ./new.txt FIELD3 ./old.txt FIELD5 FIELD6 FIELD7
sec/op CI sec/op CI vs base P
OPAEval 4.8084e-05 56% 0.5419958225 1% +1127085.39% p=0.000 n=10
geomean 4.808400000000004e-05 0.5419958225 +1127085.39%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_75% CI histogram_timer_rego_external_resolve_ns_75% CI vs base P
OPAEval 158 51% 718.5 12% +354.75% p=0.000 n=10
geomean 158 718.4999999999999 +354.75%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_90% CI histogram_timer_rego_external_resolve_ns_90% CI vs base P
OPAEval 213 47% 718.5 12% +237.32% p=0.000 n=10
geomean 213.00000000000006 718.4999999999999 +237.32%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_95% CI histogram_timer_rego_external_resolve_ns_95% CI vs base P
OPAEval 267.55 45% 718.5 12% +168.55% p=0.000 n=10
geomean 267.5500000000001 718.4999999999999 +168.55%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_99% CI histogram_timer_rego_external_resolve_ns_99% CI vs base P
OPAEval 454.75 50% 718.5 12% +58.00% p=0.000 n=10
geomean 454.75000000000017 718.4999999999999 +58.00%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_99.9% CI histogram_timer_rego_external_resolve_ns_99.9% CI vs base P
OPAEval 1343.5 367% 718.5 12% -46.52% p=0.023 n=10
geomean 1343.5000000000002 718.4999999999999 -46.52%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_99.99% CI histogram_timer_rego_external_resolve_ns_99.99% CI vs base P
OPAEval 1358.5 374% 718.5 12% -47.11% p=0.023 n=10
geomean 1358.5000000000005 718.4999999999999 -47.11%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_count CI histogram_timer_rego_external_resolve_ns_count CI vs base P
OPAEval 25967 103% 2 0% -99.99% p=0.000 n=10
geomean 25967.000000000015 2 -99.99%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_max CI histogram_timer_rego_external_resolve_ns_max CI vs base P
OPAEval 1358.5 374% 718.5 12% -47.11% p=0.023 n=10
geomean 1358.5000000000005 718.4999999999999 -47.11%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_mean CI histogram_timer_rego_external_resolve_ns_mean CI vs base P
OPAEval 159.7 53% 631.5 16% +295.43% p=0.000 n=10
geomean 159.7 631.5 +295.43%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_median CI histogram_timer_rego_external_resolve_ns_median CI vs base P
OPAEval 140 64% 631.5 16% +351.07% p=0.000 n=10
geomean 139.99999999999994 631.5 +351.07%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_min CI histogram_timer_rego_external_resolve_ns_min CI vs base P
OPAEval 82 54% 607 23% +640.24% p=0.000 n=10
geomean 82.00000000000001 607 +640.24%
./new.txt ./old.txt
histogram_timer_rego_external_resolve_ns_stddev CI histogram_timer_rego_external_resolve_ns_stddev CI vs base P
OPAEval 81.78 155% 44 245% ~ p=0.063 n=10
geomean 81.78000000000003 43.99999999999999 -46.20%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_75% CI histogram_timer_rego_query_eval_ns_75% CI vs base P
OPAEval 40758 48% 5.523685545e+08 1% +1355139.60% p=0.000 n=10
geomean 40758.000000000015 5.523685545e+08 +1355139.60%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_90% CI histogram_timer_rego_query_eval_ns_90% CI vs base P
OPAEval 60448.5 50% 5.523685545e+08 1% +913683.72% p=0.000 n=10
geomean 60448.49999999999 5.523685545e+08 +913683.72%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_95% CI histogram_timer_rego_query_eval_ns_95% CI vs base P
OPAEval 85048 54% 5.523685545e+08 1% +649378.59% p=0.000 n=10
geomean 85047.99999999994 5.523685545e+08 +649378.59%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_99% CI histogram_timer_rego_query_eval_ns_99% CI vs base P
OPAEval 131346 56% 5.523685545e+08 1% +420444.63% p=0.000 n=10
geomean 131345.9999999999 5.523685545e+08 +420444.63%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_99.9% CI histogram_timer_rego_query_eval_ns_99.9% CI vs base P
OPAEval 257017 52% 5.523685545e+08 1% +214815.18% p=0.000 n=10
geomean 257016.99999999997 5.523685545e+08 +214815.18%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_99.99% CI histogram_timer_rego_query_eval_ns_99.99% CI vs base P
OPAEval 258342.5 52% 5.523685545e+08 1% +213712.50% p=0.000 n=10
geomean 258342.4999999998 5.523685545e+08 +213712.50%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_count CI histogram_timer_rego_query_eval_ns_count CI vs base P
OPAEval 25967 103% 2 0% -99.99% p=0.000 n=10
geomean 25967.000000000015 2 -99.99%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_max CI histogram_timer_rego_query_eval_ns_max CI vs base P
OPAEval 258342.5 52% 5.523685545e+08 1% +213712.50% p=0.000 n=10
geomean 258342.4999999998 5.523685545e+08 +213712.50%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_mean CI histogram_timer_rego_query_eval_ns_mean CI vs base P
OPAEval 41433 56% 5.41978532e+08 1% +1307984.21% p=0.000 n=10
geomean 41433.00000000002 5.419785320000008e+08 +1307984.21%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_median CI histogram_timer_rego_query_eval_ns_median CI vs base P
OPAEval 34818 57% 5.41978532e+08 1% +1556504.43% p=0.000 n=10
geomean 34818.00000000002 5.419785320000008e+08 +1556504.43%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_min CI histogram_timer_rego_query_eval_ns_min CI vs base P
OPAEval 20012.5 55% 5.28732941e+08 1% +2641913.45% p=0.000 n=10
geomean 20012.500000000015 5.287329410000003e+08 +2641913.45%
./new.txt ./old.txt
histogram_timer_rego_query_eval_ns_stddev CI histogram_timer_rego_query_eval_ns_stddev CI vs base P
OPAEval 21401 48% 1.2787495e+07 30% +59651.86% p=0.000 n=10
geomean 21400.99999999998 1.2787495000000002e+07 +59651.86%
./new.txt ./old.txt
B/op CI B/op CI vs base P
OPAEval 12769 0% 2.19473014e+08 0% +1718695.63% p=0.000 n=10
geomean 12769.000000000007 2.1947301399999982e+08 +1718695.63%
./new.txt ./old.txt
allocs/op CI allocs/op CI vs base P
OPAEval 206 0% 5.2844555e+06 0% +2565169.66% p=0.000 n=10
geomean 206.00000000000003 5.284455500000005e+06 +2565169.66%
  1. 181 users benchmark
FIELD1 ./new-181.txt FIELD3 ./old-181.txt FIELD5 FIELD6 FIELD7
sec/op CI sec/op CI vs base P
OPAEval 0.000369884 2% 26.711795693 2% +7221568.33% p=0.000 n=10
geomean 0.0003698839999999999 26.711795692999996 +7221568.33%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_75% CI histogram_timer_rego_external_resolve_ns_75% CI vs base P
OPAEval 239 1% 1.2813105e+06 4% +536013.18% p=0.000 n=10
geomean 239.00000000000009 1.2813105000000007e+06 +536013.18%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_90% CI histogram_timer_rego_external_resolve_ns_90% CI vs base P
OPAEval 277.65 7% 1.2813105e+06 4% +461384.06% p=0.000 n=10
geomean 277.6499999999999 1.2813105000000007e+06 +461384.06%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_95% CI histogram_timer_rego_external_resolve_ns_95% CI vs base P
OPAEval 360.85 7% 1.2813105e+06 4% +354981.20% p=0.000 n=10
geomean 360.8500000000001 1.2813105000000007e+06 +354981.20%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_99% CI histogram_timer_rego_external_resolve_ns_99% CI vs base P
OPAEval 690.0999999999999 32% 1.2813105e+06 4% +185570.27% p=0.000 n=10
geomean 690.0999999999997 1.2813105000000007e+06 +185570.27%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_99.9% CI histogram_timer_rego_external_resolve_ns_99.9% CI vs base P
OPAEval 2262.5 117% 1.2813105e+06 4% +56532.51% p=0.000 n=10
geomean 2262.499999999999 1.2813105000000007e+06 +56532.51%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_99.99% CI histogram_timer_rego_external_resolve_ns_99.99% CI vs base P
OPAEval 2288.5 119% 1.2813105e+06 4% +55889.10% p=0.000 n=10
geomean 2288.4999999999995 1.2813105000000007e+06 +55889.10%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_count CI histogram_timer_rego_external_resolve_ns_count CI vs base P
OPAEval 3052 8% 1 0% -99.97% p=0.000 n=10
geomean 3052 1 -99.97%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_max CI histogram_timer_rego_external_resolve_ns_max CI vs base P
OPAEval 2288.5 119% 1.2813105e+06 4% +55889.10% p=0.000 n=10
geomean 2288.4999999999995 1.2813105000000007e+06 +55889.10%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_mean CI histogram_timer_rego_external_resolve_ns_mean CI vs base P
OPAEval 250.15 2% 1.2813105e+06 4% +512116.87% p=0.000 n=10
geomean 250.14999999999998 1.2813105000000007e+06 +512116.87%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_median CI histogram_timer_rego_external_resolve_ns_median CI vs base P
OPAEval 231 2% 1.2813105e+06 4% +554579.87% p=0.000 n=10
geomean 230.99999999999994 1.2813105000000007e+06 +554579.87%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_min CI histogram_timer_rego_external_resolve_ns_min CI vs base P
OPAEval 195 1% 1.2813105e+06 4% +656982.31% p=0.000 n=10
geomean 195 1.2813105000000007e+06 +656982.31%
./new-181.txt ./old-181.txt
histogram_timer_rego_external_resolve_ns_stddev CI histogram_timer_rego_external_resolve_ns_stddev CI vs base P
OPAEval 111.175 55% 0 0% -100.00% p=0.000 n=10
geomean 111.17499999999995 ?
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_75% CI histogram_timer_rego_query_eval_ns_75% CI vs base P
OPAEval 374831.5 1% 2.67117748435e+10 2% +7126242.06% p=0.000 n=10
geomean 374831.5000000002 2.671177484349996e+10 +7126242.06%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_90% CI histogram_timer_rego_query_eval_ns_90% CI vs base P
OPAEval 397388 2% 2.67117748435e+10 2% +6721737.31% p=0.000 n=10
geomean 397387.99999999977 2.671177484349996e+10 +6721737.31%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_95% CI histogram_timer_rego_query_eval_ns_95% CI vs base P
OPAEval 455126 9% 2.67117748435e+10 2% +5868994.46% p=0.000 n=10
geomean 455126 2.671177484349996e+10 +5868994.46%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_99% CI histogram_timer_rego_query_eval_ns_99% CI vs base P
OPAEval 877206.5 30% 2.67117748435e+10 2% +3044995.41% p=0.000 n=10
geomean 877206.4999999997 2.671177484349996e+10 +3044995.41%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_99.9% CI histogram_timer_rego_query_eval_ns_99.9% CI vs base P
OPAEval 1.0866645e+06 704% 2.67117748435e+10 2% +2458043.69% p=0.000 n=10
geomean 1.086664499999999e+06 2.671177484349996e+10 +2458043.69%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_99.99% CI histogram_timer_rego_query_eval_ns_99.99% CI vs base P
OPAEval 1.0881685e+06 724% 2.67117748435e+10 2% +2454646.19% p=0.000 n=10
geomean 1.0881684999999993e+06 2.671177484349996e+10 +2454646.19%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_count CI histogram_timer_rego_query_eval_ns_count CI vs base P
OPAEval 3052 8% 1 0% -99.97% p=0.000 n=10
geomean 3052 1 -99.97%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_max CI histogram_timer_rego_query_eval_ns_max CI vs base P
OPAEval 1.0881685e+06 724% 2.67117748435e+10 2% +2454646.19% p=0.000 n=10
geomean 1.0881684999999993e+06 2.671177484349996e+10 +2454646.19%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_mean CI histogram_timer_rego_query_eval_ns_mean CI vs base P
OPAEval 365809.5 1% 2.67117748435e+10 2% +7301999.82% p=0.000 n=10
geomean 365809.4999999999 2.671177484349996e+10 +7301999.82%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_median CI histogram_timer_rego_query_eval_ns_median CI vs base P
OPAEval 345961 2% 2.67117748435e+10 2% +7720936.43% p=0.000 n=10
geomean 345961.00000000006 2.671177484349996e+10 +7720936.43%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_min CI histogram_timer_rego_query_eval_ns_min CI vs base P
OPAEval 293570.5 1% 2.67117748435e+10 2% +9098830.19% p=0.000 n=10
geomean 293570.5000000002 2.671177484349996e+10 +9098830.19%
./new-181.txt ./old-181.txt
histogram_timer_rego_query_eval_ns_stddev CI histogram_timer_rego_query_eval_ns_stddev CI vs base P
OPAEval 80031 247% 0 0% -100.00% p=0.000 n=10
geomean 80031.00000000007 ?
./new-181.txt ./old-181.txt
B/op CI B/op CI vs base P
OPAEval 128446.5 0% 6.489615516e+09 0% +5052287.97% p=0.000 n=10
geomean 128446.50000000006 6.489615515999995e+09 +5052287.97%
./new-181.txt ./old-181.txt
allocs/op CI allocs/op CI vs base P
OPAEval 2884 0% 2.426877455e+08 0% +8414870.37% p=0.000 n=10
geomean 2884.0000000000014 2.426877455000003e+08 +8414870.37%

@ZohebShaikh ZohebShaikh requested a review from tpoliaw February 2, 2026 13:33
tpoliaw
tpoliaw requested changes Feb 4, 2026
View reviewed changes
Copy link
Collaborator

@tpoliaw tpoliaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can make out, this looks reasonable and you can't argue with a 13,000x speedup.

It could really do with some comments though on things like the difference between _session, user_session and user_sessions so the next person who looks at this doesn't have to figure it out from scratch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tpoliaw tpoliaw tpoliaw requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ZohebShaikh @tpoliaw