Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproducible random does not work in trip scheduling choice #891

Open
jpn-- opened this issue Sep 25, 2024 · 1 comment
Open

Reproducible random does not work in trip scheduling choice #891

jpn-- opened this issue Sep 25, 2024 · 1 comment
Labels
Bug Something isn't working/bug f

Comments

@jpn--
Copy link
Member

jpn-- commented Sep 25, 2024

Describe the bug
The reproducible random code is supposed to ensure that, when run with a fixed seed, household-relevant results (persons, trips, tours) for any particular household remain stable no matter how many other households are in the sample. This is not the case for the trip scheduling choice model (currently used only by ARC).

Note this bug does not impact the statistical distribution or validity of results, it merely makes model testing problematic.

To Reproduce
Steps to reproduce the behavior:

  1. Run the ARC model single process for some moderate sized household sample.
  2. Run the ARC model multi-process, with num_processes > 1 for the same household sample. Some households will be processes in different groups, which should not change the results.
  3. Observe that a small percentage of household trip schedules differ, typically by a small amount.
- Column 'main_leg_duration' failure
AssertionError: Series are different

Series values are different (22.66227 %)
[index]: [18413, 18448, 18473, 18489, 30371, 66957, 66992, 66998, 67033, 67066, 67104, 67148, 88747, 88763, 127869, 127871, 271675, 271676, 271683, 271746, 271779, 271820, 391140, 391142, 391179, 391214, 391226, 391261, 410255, 410256, 410359, 410378, 410443, 533935, 533941, 541799, 541846, 543947, 543986, 544019, 578877, 633385, 633407, 633437, 633442, 633489, 633522, 633530, 656794, 656812, 656853, 656854, 656867, 656892, 721604, 721609, 721639, 721669, 721713, 721754, 721795, 779015, 779028, 779066, 788961, 811612, 811620, 811631, 811664, 977162, 977230, 977233, 977260, 1031632, 1031673, 1178586, 1178601, 1178625, 1178636, 1178637, 1178660, 1178677, 1178740, 1224715, 1224750, 1224791, 1224792, 1224824, 1234344, 1234377, 1234418, 1254855, 1254856, 1254857, 1254912, 1254926, 1254956, 1254959, 1255000, 1274401, ...]
[left]:  [0, 20, 4, 21, 19, 3, 16, 1, 21, 23, 6, 11, 1, 24, 13, 2, 0, 1, 0, 19, 17, 23, 2, 2, 15, 2, 2, 18, 2, 2, 17, 31, 4, 1, 3, 4, 14, 5, 16, 26, 21, 0, 21, 1, 3, 17, 5, 22, 6, 3, 3, 2, 4, 21, 1, 0, 9, 2, 17, 24, 16, 8, 4, 4, 22, 4, 1, 7, 0, 0, 6, 18, 5, 15, 8, 9, 5, 10, 2, 4, 2, 0, 19, 3, 21, 20, 14, 16, 16, 16, 16, 4, 1, 1, 4, 24, 2, 15, 23, 15, ...]
[right]: [0, 20, 4, 19, 19, 3, 16, 1, 21, 23, 6, 11, 1, 24, 15, 2, 0, 1, 0, 20, 17, 23, 2, 2, 15, 2, 2, 18, 2, 3, 17, 31, 4, 1, 3, 4, 14, 5, 11, 26, 21, 0, 21, 1, 5, 14, 5, 22, 6, 3, 3, 1, 4, 18, 1, 0, 8, 2, 17, 24, 16, 8, 4, 4, 21, 4, 2, 7, 1, 0, 6, 20, 5, 15, 8, 9, 5, 13, 2, 4, 2, 0, 19, 3, 21, 20, 13, 16, 16, 16, 16, 4, 1, 1, 4, 24, 2, 15, 23, 15, ...]

Expected behavior
Multiprocessing should not alter the results.

@jpn-- jpn-- added the Bug Something isn't working/bug f label Sep 25, 2024
@guyrousseau
Copy link

Thanks @jpn--, as an FYI, for more details on the aforementioned ARC trip scheduling choice model / intermediate stop duration timing model development effort, see https://onlinepubs.trb.org/onlinepubs/conferences/2014/ITM/Resources/78.pdf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working/bug f
Projects
Status: No status
Development

No branches or pull requests

2 participants