Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
852 commits
Select commit Hold shift + click to select a range
7aff81f
nits: updating checks to every 1s instead of 0.1s
dhruvsgarg Oct 21, 2024
d5e1d24
Testing with lowered concurrency and agg goal
dhruvsgarg Oct 25, 2024
ff3ef03
Supporting round_nudge_type as train or eval based on async_oort option
Oct 26, 2024
21b27b3
config for supporting eval based rank nudging in asyncoort
Oct 26, 2024
20b118b
More logging to track utility and staleness
Oct 26, 2024
b9df28e
cleaning up some logging
Oct 26, 2024
b35ad2e
Deleted cifar 2-state configs for dir100 num300 traceFail 6d oort
Oct 26, 2024
728097b
new traceFail 6day configs with 3states for all alphas for with and w…
Oct 27, 2024
5546588
script to copy trainer_indices_list from old configs to new
Oct 27, 2024
c4992fb
[cifar_n300] scripts and agg jsons for first set of runs
Oct 27, 2024
a280d5e
minor fix in bash script
dhruvsgarg Oct 27, 2024
beee513
updated logging in asyncoort; reduced evalFactor to 1
dhruvsgarg Oct 27, 2024
6932d5f
rename bash files to reflect reduced evalGoal of 1
dhruvsgarg Oct 27, 2024
5e1c477
fix: slow gpu model loads caused mqtt timeouts and incorrect channel …
Oct 27, 2024
185eecb
updated script for cifar run (jayne) fedbuff asyncoort clientNotify
Oct 27, 2024
fb0119f
updated iAgg iSel clientNotify jsons and scripts for different eval f…
Oct 27, 2024
acf73a8
fixed eval selection policy and concurrency in asyncoort
Oct 28, 2024
96f7400
updating eval selector to only operate on AVL_EVAL trainers
Oct 28, 2024
8b2acdf
adding speedup factor in async cifar10 trainer to quicken up runs
Oct 28, 2024
1b289b2
updating configs and scripts to use speedup
Oct 28, 2024
adc3c77
add script to test felix with eval goal 0
Oct 28, 2024
fa102cc
re-adding avl_train selectors in filtered ends for eval task
Oct 29, 2024
2f1698b
configs for felix and baseline expts
Oct 29, 2024
d5942f8
fixed alpha values in the script for felix runs
Oct 29, 2024
537a205
hot-fix: handle_recv_state would block on trainers and start listenin…
Oct 29, 2024
d5bf328
continuing the handle_recv_state hotfix
Oct 29, 2024
6c98972
adding config for kaylee fedbuff asyncoort clientnotify alphas 0.1 an…
Oct 29, 2024
c156e59
adding additional log for last eval round and stat utility
Oct 31, 2024
d173bd3
NRL: mqtt issue analysis
Nov 21, 2024
5c57100
added log lines
Nov 24, 2024
274378a
code backup
Dec 3, 2024
da6f247
pushing overdue mlsys25 scripts
Jan 28, 2025
f4bf333
Update setup.py to build flame with python 3.7.15 (fwdllm req)
Jan 29, 2025
23d5aed
NRL: modified gitignore
nehalakhani1805 Jan 31, 2025
d29a2f7
NRL: loaded local data for aggregator and each server
nehalakhani1805 Jan 31, 2025
9e6b29e
flash_trace_analysis notebook
dhruvsgarg Feb 5, 2025
54debb5
mlsys25 plotting script
dhruvsgarg Feb 5, 2025
eddfae7
code changes to make syncfl oort run with mobiperf traceavail
dhruvsgarg Feb 6, 2025
72fa9aa
temp config changes to run syncfl oort expt
dhruvsgarg Feb 6, 2025
47f68e9
updated other 3 heterogeneity configs for syncfl oort expt
dhruvsgarg Feb 6, 2025
1443c9b
increase agg_num to meet oort paper 8% participation ratio
dhruvsgarg Feb 6, 2025
1f95e34
added bash scripts for starting trainers
dhruvsgarg Feb 6, 2025
35fbb0e
added scripts to split mobiperf syncfl oort expt across 2 nodes
dhruvsgarg Feb 6, 2025
4bbc061
fixes to syncfl oort selector
dhruvsgarg Feb 6, 2025
9bb2034
minor fix: init of round_duration
dhruvsgarg Feb 6, 2025
170e642
fixes again: oort round duration
dhruvsgarg Feb 6, 2025
3fd1190
updated names for expts
dhruvsgarg Feb 6, 2025
60529c0
lowering common communication log statements from info to debug
Feb 7, 2025
93f709b
Update flame init to log only at warning level in /tmp/
Feb 7, 2025
cebc00c
revert log level to info but removed the log_path to /tmp
Feb 7, 2025
d514d27
removed /tmp log file handler for flame, only stdout
Feb 7, 2025
e5ae8f5
updated expt files for syncfl oort
Feb 7, 2025
c6e3793
fixed 2 trainer configs with empty data lists in alpha=0.1 num=300
dhruvsgarg Feb 7, 2025
146cc03
NRL: MQTT setup
nehalakhani1805 Feb 7, 2025
f275afb
init changes to attempt asyncfl top agg to use oracular trace
dhruvsgarg Feb 8, 2025
a9f7af0
added few debug statements
dhruvsgarg Feb 8, 2025
0c9be6a
added scripts and json for asyncoort oracular run
dhruvsgarg Feb 8, 2025
df5bb8d
bash scripts with reduced sleep interval between trainer starts
dhruvsgarg Feb 8, 2025
de5121b
scripts for asyncoort expts
dhruvsgarg Feb 8, 2025
b2eb51f
added 20 % failures
riksanyal Feb 9, 2025
75cee1a
added 40% failures
riksanyal Feb 9, 2025
663bdd8
20% and 40% failures
riksanyal Feb 9, 2025
a6d9363
add 120 secs avail at the beginning
riksanyal Feb 9, 2025
98779bd
adding script for asyncoort synthetic failure 20% run
dhruvsgarg Feb 9, 2025
8a89abf
set agg to fetch oracular trainer info for 20% synthetic failure dist…
dhruvsgarg Feb 9, 2025
2035935
updating main.py agg async_cifar10 for synthetic 20% expt
dhruvsgarg Feb 9, 2025
1bf05dc
fix data format issue in trainer json files avl_events_syn_20
dhruvsgarg Feb 9, 2025
291e61d
more fixes to synthetic trainer failure 20% json field
dhruvsgarg Feb 9, 2025
69f1df9
updating oort agg for syn avail trace, adding bash scripts for 20% an…
dhruvsgarg Feb 9, 2025
b7c8a3e
fix names in 40% trace fail syn bash scripts
Feb 9, 2025
92a5fd5
higher concurrency configs and scripts for synthetic expts
dhruvsgarg Feb 9, 2025
d97935a
updating syncfl oort server lr for fedavg based on google keyboard pa…
dhruvsgarg Feb 10, 2025
d58d042
making server lr for syncfl oort same as asyncoort
dhruvsgarg Feb 10, 2025
5bf5ee0
changing synthetic failure notebook for new config style
riksanyal Feb 10, 2025
e79810d
lowering syncfl c to match 8% literature value
Feb 10, 2025
12db665
moving synthetic fail script from alpha 100 to 0.1
Feb 10, 2025
a20297a
updating asyncoort to operate at c60 to match 20% (same as fedbuff c)
Feb 10, 2025
e2c8a6e
updating concurrency for sync and async in wandb name
Feb 10, 2025
c830495
updated logging in oort to track staleness and utility
dhruvsgarg Feb 10, 2025
114ba1c
updated configs to run 100% avail syn expt
dhruvsgarg Feb 10, 2025
4aa89f7
updated scripts for 100% avail expts
dhruvsgarg Feb 10, 2025
b94c028
agg and agg_oort will always believe everything is available
dhruvsgarg Feb 10, 2025
1151adf
small fix in bash script for 0fail syn expt
dhruvsgarg Feb 10, 2025
646994d
fixed nit in oort about agg finish log line
Feb 10, 2025
01c5ec1
fixing the updates_recv tracking for oort top agg
Feb 10, 2025
6da1bba
fixed names for expt runs after lowering back the concurrency
Feb 10, 2025
2ba26a8
added 0%, 20% and 40% syn unavail to all num300_traceFail_6d_oort dirs
Feb 11, 2025
83abf8f
init impl for disabling clientNotify but enable failures on trainer
Feb 11, 2025
cd98d8e
updating trainer main.py to work with synthetic failure traces
Feb 11, 2025
7e8285f
script for unaware asyncoort alpha100 num300 syn_20
dhruvsgarg Feb 11, 2025
918f1eb
init agg json to try asyncoort unaware syn_20 alpha100 num300
Feb 11, 2025
d06513a
update trainer to keep updating local trainer state even if clientNot…
Feb 11, 2025
cbe01d9
prep for unaware expts: updating all oort configs to two_state and wa…
dhruvsgarg Feb 11, 2025
8780e15
bash script and agg json for asyncoort mobiperf unaware alpha100_0.1 …
dhruvsgarg Feb 11, 2025
a1462a8
revert to three_state avail for mobiperf unaware expts
dhruvsgarg Feb 11, 2025
14d7284
script for mobiperf unaware oort alpha100_0.1 num300
Feb 11, 2025
4554cb0
fix json name for mobiperf unaware oort
Feb 11, 2025
1c37c78
increase timeouts for mqtt to 4hours
dhruvsgarg Feb 11, 2025
4d04722
fix the agg json for oort unaware mobiperf
Feb 11, 2025
61fab2c
02/11 tmp
Feb 11, 2025
218c74d
Merge branch 'dump' into dg/new_event_state_configs
Feb 11, 2025
8bafe1a
scripts and configs for synthetic unaware 40% unavail, oort and async…
Feb 11, 2025
256287a
euromlsys25 plots and plotting script
Feb 12, 2025
af969c2
Merge branch 'dg/new_event_state_configs' (after workshop deadline) i…
dhruvsgarg Feb 17, 2025
65bc606
design changes for FedSGDTrainer
nehalakhani1805 Feb 18, 2025
f50cb34
agg sends weights, trainer fetches weights
nehalakhani1805 Feb 21, 2025
8b25744
removed log line
nehalakhani1805 Feb 21, 2025
ac0943c
checking device for clients and agg
nehalakhani1805 Feb 21, 2025
fcfcd54
sent trainers and agg to cuda devices
nehalakhani1805 Feb 22, 2025
710bd44
run black formatter on repo (except possibly conflicting fwdllm files)
dhruvsgarg Feb 22, 2025
66eb147
delete commented out things in fwdllm agg; update bash script for fwd…
dhruvsgarg Feb 22, 2025
db6f26d
trainers train and send grads to agg
nehalakhani1805 Feb 23, 2025
6af634f
nit: formatting some files
dhruvsgarg Feb 23, 2025
0cbb347
fix: sending gradients from trainer to agg
dhruvsgarg Feb 27, 2025
05f1be1
var check on agg side
nehalakhani1805 Mar 1, 2025
0805604
send gradients in correct shape for update and var check
dhruvsgarg Mar 2, 2025
73d3e7a
aggregate if var good enough
nehalakhani1805 Mar 8, 2025
c6fc066
added new trainer json
nehalakhani1805 Mar 8, 2025
8f81fd2
added more trainers + selector bug fixes
nehalakhani1805 Mar 10, 2025
2323091
config changes
nehalakhani1805 Mar 10, 2025
499f808
[EuroMLSys'25] final plotting script and plots
Mar 12, 2025
4ea7ff5
[fwdllm] fix: allow trainer to rerun in the same round w/ same weights
Mar 24, 2025
67774e2
fixed caching for all trainers, defer selection until all trainers ar…
nehalakhani1805 Mar 31, 2025
fa4dfae
forgot to save
nehalakhani1805 Mar 31, 2025
1d8f795
nits, moving from sheph to wash
dhruvsgarg Apr 5, 2025
f43130b
[fwdllm] eval_model() on gpu
Apr 5, 2025
7742c83
changes for one update per iteration + logging for debugging
nehalakhani1805 Apr 11, 2025
3e4df2c
repartitioning + cuda oom reduction
nehalakhani1805 Apr 14, 2025
68596d5
[fwdllm] Verifying gpu allocation per trainer
Apr 14, 2025
3771f54
[fwdllm] better view of the gpu memory issue
Apr 14, 2025
b56101e
[fwdllm] Pushing neha's 14-apr night changes to reduce memory
Apr 15, 2025
792e202
oom reduction
nehalakhani1805 Apr 18, 2025
ccc85ad
[fwdllm] scaling up tc_expt to 100 trainers
dhruvsgarg Apr 26, 2025
f8adb46
[fwdllm] incorporating fixes, scale up to 100 trainers
dhruvsgarg Apr 27, 2025
3dc2c29
[fwdllm] tried fixes to agg, process still killed due to oom
dhruvsgarg Apr 28, 2025
f38fe66
adding logs for trainer aggregator cpu mem occupied too
dhruvsgarg Apr 29, 2025
c634622
added cpu mem fix in agg
dhruvsgarg Apr 30, 2025
895b0c1
[fwdllm] breaking changes, trying to reduce mqtt message transfer and…
dhruvsgarg Apr 30, 2025
c1621b4
reduced weights size + fixed grad 0
nehalakhani1805 May 1, 2025
4cfadb2
[fwdllm] fixed a few things to run without large mqtt messages
dhruvsgarg May 1, 2025
8126a11
[fwdllm] retrying with agg_goal equals k
dhruvsgarg May 1, 2025
5c39969
[fwdllm] keeping concurrency to 100 trainers, not 95
dhruvsgarg May 1, 2025
ad2556c
[fwdllm] optimized memory alloc for trainer
May 1, 2025
e578ae4
added grad pool back and fixed some logic
nehalakhani1805 May 2, 2025
493dfc3
[fwdllm] wip: attempting fix for grad_pool changes
May 2, 2025
3a7b35e
[fwdllm] reverting to 100 trainer expt
May 3, 2025
4a464b0
[fwdllm] more fixes, reaches 65 acc
May 4, 2025
2dd059a
v_buffer fix
nehalakhani1805 May 5, 2025
3efe0fd
Merging working version of fwdllm_flame into dg/new_event_state_configs
dhruvsgarg May 6, 2025
96d2436
reorg client behavior traces
May 6, 2025
2cc18c5
added jsons for synthetic 50% trainer avail
May 7, 2025
9def5fc
reduced logs at info level to produce sparser logs for e2e expts
May 7, 2025
a7be755
lower level agg and select telemetry, needs testing
May 8, 2025
9e1f757
reverting data dir to use local dataset (same node, disk)
May 9, 2025
f924b2c
uppdates to scripts, configs for expts
May 10, 2025
3d6af2d
added script for cifar oort_oracular runs
May 10, 2025
d1d2078
updated script for sync oort oracular
May 10, 2025
3bf02a5
working changes for oortAsync
May 10, 2025
c4e8844
update json and code for client_notify "type" to "trace"
May 10, 2025
941b192
adding selector stats to fedbuff and oort
May 10, 2025
a6e15ca
fixing oort configs
May 11, 2025
567ec97
bugfix: add check to see if av_events are left to process
May 11, 2025
9effa4c
init scripts for felix runs with automated json updates
dhruvsgarg May 11, 2025
d183a36
slight fixes to run felix expt
dhruvsgarg May 11, 2025
7d03fbf
nit fix in felix json for cifar expt
dhruvsgarg May 11, 2025
5b408fe
use conda env to make jq available
dhruvsgarg May 11, 2025
6eb0628
felix trainer start json script, new oortAsync scripts
dhruvsgarg May 11, 2025
f6f6808
try modified eval to fix felix
dhruvsgarg May 13, 2025
55ade4d
fix eval selector criteria and add felix scripts
dhruvsgarg May 13, 2025
04fb0a1
config and script for felix abl, no tierFuse
dhruvsgarg May 13, 2025
0a457bb
felix ablation without eval selector and tierfuse
dhruvsgarg May 13, 2025
fad3c00
pushing nehas code up
May 13, 2025
eed06ab
pushing plotting code and cifar plots
May 14, 2025
d24a0bb
adding scripts for two synthetic oort unaware runs
May 14, 2025
c5a96d0
felix ablation run to see benefits of evalSelector
May 14, 2025
c3d1eda
nit fix to oort selector
May 14, 2025
c36ce01
fixes or oort unaware
dhruvsgarg May 14, 2025
ae12e96
nit fix to bash scripts for oort unaware
dhruvsgarg May 14, 2025
9667975
undo fix in trainer notify, only asyncoort uses notifs
dhruvsgarg May 15, 2025
a7b252e
updates to plots and pdfs with oort (unaware)
May 15, 2025
b24fe91
scripts and config for oortAsync unaware
May 15, 2025
ab330a8
updated script for oortAsync unaware run
May 15, 2025
4b44231
plot script with stat util cdf
May 15, 2025
bce7edf
selector stats: cdfs for stat util and speed
May 15, 2025
036f24d
pushing current changes in plotting script
May 15, 2025
9d44445
latest plotting script
May 16, 2025
883a3d7
Env setup (#5)
gdadlaney May 29, 2025
6ef1133
Env setup for FwdLLM (#6)
gdadlaney Jun 7, 2025
8d4b44a
Experiment 2: Log training data at clients (#7)
gdadlaney Aug 18, 2025
621c2ee
Fix the path of the data partition files in the config.json files and…
nehalakhani1805 Sep 8, 2025
9d7449c
Merge pull request #9 from dhruvsgarg/fix/run_config
gdadlaney Sep 9, 2025
9de7414
[needs cleanup] pushing plots from eurosys26 submission (#12)
dhruvsgarg Sep 24, 2025
2c58f9e
[FwdLLM] Fix fwdllm selectors (#10)
gdadlaney Oct 1, 2025
2c5f5a4
[FwdLLM] Generate 10 candidate perturbations & only select the top 1 …
gdadlaney Oct 1, 2025
32dc5c7
[FwdLLM] Add logs to compare the Perf issue (#8)
gdadlaney Oct 1, 2025
41dda72
Quick fixes to resolve the incorrectly resolved merge conflicts
gdadlaney Oct 2, 2025
bad14a2
Merge pull request #13 from dhruvsgarg/quick_fix
gdadlaney Oct 6, 2025
553a53b
update model version per model update
AishwwaryaM Nov 11, 2025
3c5afe6
Trainer and aggregator changes for all baselines
AishwwaryaM Nov 12, 2025
3a0c20d
revert unreviewed changes (#23)
AishwwaryaM Nov 12, 2025
3e75149
Handle unavailability of trainers and aggregator (#24)
AishwwaryaM Nov 12, 2025
9793788
bug fix, return only selected candidates (#25)
AishwwaryaM Nov 13, 2025
e44a277
Async fwdllm v1 [WIP] (#18)
ShreyaChivilkar Nov 14, 2025
32f5ed4
Python scripts to parse logs & create visualizations (#16)
gdadlaney Nov 19, 2025
9e82a58
Async fwdllm: Weighted aggregation (#27)
gdadlaney Jan 9, 2026
1eb4ee6
Fix log level in logging (#28)
gdadlaney Jan 12, 2026
4165d4d
Fixed regression issues on run_text_classification.sh (#30)
gdadlaney Jan 28, 2026
633c12e
Formatting files (no logic changes) (#31)
dhruvsgarg Jan 30, 2026
b31a531
Programmatic Experiment Generation for async_cifar10 (#32)
dhruvsgarg Feb 4, 2026
4b63ef8
Improving eval time perf on Aggregator (#35)
gdadlaney Feb 23, 2026
ab4cad6
Update-databin-used-for-training (#37)
AishwwaryaM Feb 23, 2026
bc0c43c
Async with Performance optimizations (#39)
gdadlaney Mar 2, 2026
11da883
Add debug logs for sanity checks (#38)
AishwwaryaM Mar 2, 2026
2bcce1d
Add Random async selector (#41)
AishwwaryaM Mar 7, 2026
07fc37c
Improving train time perf (#34)
gdadlaney Mar 9, 2026
9503cad
[SoCC'26] Refreshed REFL expts with sanity checks, scripts and plots …
dhruvsgarg Mar 9, 2026
0fae306
add timeout and fix stalling (#42)
AishwwaryaM Mar 9, 2026
13f9599
[refl, oort] nit cleanup changes (#45)
dhruvsgarg Mar 10, 2026
94fd02a
Add 3 state synthetic traces for FwdLLM Configs (#47)
AishwwaryaM Mar 10, 2026
a689cde
Quick additions for Async_FwdLLM run (#46)
gdadlaney Mar 10, 2026
6565d94
add traces to these so that oracular works (#48)
AishwwaryaM Mar 10, 2026
238c287
Change heavy loggers to DEBUG (#50)
gdadlaney Mar 14, 2026
4592c01
Script to stop the run after a certain accuracy has been reached N co…
gdadlaney Mar 14, 2026
c5bca20
Fix the hardcode on file names used when using Oracular knowledge (#51)
gdadlaney Mar 16, 2026
c3a727f
Fix Async OORT selector for FwdLLM (#52)
gdadlaney Mar 16, 2026
eb09989
Fix OORT selection logic FINALLY! (#53)
gdadlaney Apr 9, 2026
dc2a746
Fix major bug in _distribute_weights() to support iteration-level sel…
gdadlaney Apr 9, 2026
9b48ea9
first cut of design, impl and tests for dynamic kc for fluxtune
dhruvsgarg Apr 12, 2026
e0648b4
changes to run init dyn k c motiv expts
dhruvsgarg Apr 14, 2026
b21137e
fixes to get dynamic kc run to execute
dhruvsgarg Apr 15, 2026
9c0a51a
switching to heterogeneous alpha 1 for run
dhruvsgarg Apr 15, 2026
e7d7f20
fix for forced agg due to max-iterations, improved log explanations f…
dhruvsgarg Apr 15, 2026
f55ce13
set var=0.3 from json for heterogeneous runs
dhruvsgarg Apr 15, 2026
40cf8b6
logical to phy mapping bug fix
dhruvsgarg Apr 18, 2026
cbb067e
scripts to launch runs in parallel
dhruvsgarg Apr 24, 2026
52daccc
cleanup comments and code for PR
dhruvsgarg Apr 25, 2026
fb3f865
future todo to fix
dhruvsgarg Apr 25, 2026
2db1116
Merge pull request #58 from dhruvsgarg/dg/dyn_k_c_asyncOORT
AishwwaryaM Apr 25, 2026
8449fac
Arm/jvp based selection (#57)
AishwwaryaM May 6, 2026
4f172c4
Dynamic KC additions (#59)
gdadlaney May 6, 2026
c885ae0
YAML launcher + shared metadata, baseline catalog, FedDance impl (#60)
dhruvsgarg May 22, 2026
5ffed03
Streaming data, more telemetry, init simulation implementation (#61)
dhruvsgarg May 30, 2026
74b17f4
Fix sim/real parity: unified max(gpu,D) timing model + GPU contention…
dhruvsgarg Jun 1, 2026
98483e6
async_cifar10: real-sim parity analysis, bug fixes, and parity checke…
dhruvsgarg Jun 8, 2026
f05bdeb
Real/sim parity: simulator fidelity fixes + parity checker (#65)
dhruvsgarg Jun 15, 2026
89a48f0
Real/sim parity: oort/refl/feddance fidelity + felix close-out + desi…
dhruvsgarg Jun 25, 2026
2dd33c2
Trainer Delay Task
TummalaSeshu Jun 29, 2026
b01408b
fwdllm: confirm launcher-migration readiness via 3 real n=100 runs (#68)
AishwwaryaM Jul 2, 2026
50c9d89
FL simulator client-unavailability substrate + parity ladder (async_c…
dhruvsgarg Jul 2, 2026
4791fba
Merge branch 'dg-fork-main' into trainer_delay_sine
TummalaSeshu Jul 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
33 changes: 33 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
name: Bug report
about: Create a report to help us improve
title: "[BUG]"
labels: ''
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**
- OS: [e.g. Linux]
- Distribution: [e.g. Ubuntu]
- Kernel Version: [e.g. 4.15.0-54]
- Software Version or Commit Hash: [e.g. 0.1.0 or 1c1ed5eb10c2070f2497a7c1cf512a25675a5263]

**Additional context**
Add any other context about the problem here.
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: '[Feature]'
labels: ''
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.
12 changes: 12 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
version: 2
updates:
- package-ecosystem: "gomod"
directory: "/"
schedule:
interval: "daily"
open-pull-requests-limit: 10
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
open-pull-requests-limit: 10
75 changes: 75 additions & 0 deletions .github/settings.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
repository:
# See https://developer.github.com/v3/repos/#edit for all available settings.

# The name of the repository. Changing this will rename the repository
name: flame

# A short description of the repository that will show up on GitHub
description: flame is a federated learning system for edge with flexibility and scalability at the core of its design.

# A URL with more information about the repository
homepage: https://www.portshift.io/blog/kubernetes-runtime-vulnerabilities-scanner-launch/

# Updates the default branch for this repository.
default_branch: main

# Either `true` to enable automated security fixes, or `false` to disable
# automated security fixes.
enable_automated_security_fixes: true

# Either `true` to enable vulnerability alerts, or `false` to disable
# vulnerability alerts.
enable_vulnerability_alerts: true

# See https://docs.github.com/en/rest/reference/teams#add-or-update-team-repository-permissions for available options
teams:
- name: flame-admins
# The permission to grant the team. Can be one of:
# * `pull` - can pull, but not push to or administer this repository.
# * `push` - can pull and push, but not administer this repository.
# * `admin` - can pull, push and administer this repository.
# * `maintain` - Recommended for project managers who need to manage the repository without access to sensitive or destructive actions.
permission: admin

- name: flame-maintainers
permission: maintain

- name: flame
permission: triage

# Collaborators: give specific users access to this repository.
# See https://docs.github.com/en/rest/reference/collaborators for available options
collaborators: []

branches:
- name: main
# https://docs.github.com/en/rest/reference/repos#update-branch-protection
# Branch Protection settings. Set to null to disable
protection:
# Required. Require at least one approving review on a pull request, before merging. Set to null to disable.
required_pull_request_reviews:
# The number of approvals required. (1-6)
required_approving_review_count: 1
# Dismiss approved reviews automatically when a new commit is pushed.
dismiss_stale_reviews: true
# Blocks merge until code owners have reviewed.
require_code_owner_reviews: true
# Specify which users and teams can dismiss pull request reviews. Pass an empty dismissal_restrictions object to disable. User and team dismissal_restrictions are only available for organization-owned repositories. Omit this parameter for personal repositories.
dismissal_restrictions:
users: []
teams: []
# Required. Require status checks to pass before merging. Set to null to disable
required_status_checks:
# Required. Require branches to be up to date before merging.
strict: true
# Required. The list of status checks to require in order to merge into this branch
contexts: []
# Required. Enforce all configured restrictions for administrators. Set to true to enforce required status checks for repository administrators. Set to null to disable.
enforce_admins: true
# Prevent merge commits from being pushed to matching branches
required_linear_history: true
# Required. Restrict who can push to this branch. Team and user restrictions are only available for organization-owned repositories. Set to null to disable.
restrictions:
apps: []
users: []
teams: []
75 changes: 75 additions & 0 deletions .github/workflows/presubmits.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Presubmits

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@f086349bfa2bd1361f7909c78558e816508cdc10 # v1
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v2

- name: Set up Go
uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v3.0.0
with:
go-version: 1.22.x

- name: Build
run: go build -v ./...

test:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@f086349bfa2bd1361f7909c78558e816508cdc10 # v1
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v2

- name: Set up Go
uses: actions/setup-go@cdcb36043654635271a94b9a6d1392de5bb323a7 # v3.0.0
with:
go-version: 1.22.x

- name: Start MongoDB
uses: supercharge/mongodb-github-action@1.11.0

- name: Test
run: go test -v -race -covermode=atomic -coverprofile=coverage.out ./...
- name: Upload codecoverage
uses: codecov/codecov-action@125fc84a9a348dbcf27191600683ec096ec9021c # 2.1.0
with:
files: ./coverage.out
verbose: true

lint:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@f086349bfa2bd1361f7909c78558e816508cdc10 # v1
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: 1.22.x

- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v1.58
45 changes: 45 additions & 0 deletions .github/workflows/publish.yml.old
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Publish

on:
release:
types: [published]

permissions:
contents: read

jobs:

build-and-push:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

-
name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@57396166ad8aefe6098280995947635806a0e6ea
with:
images: ciscoresearch/flame

-
name: Build and push
uses: docker/build-push-action@v3
with:
context: .
file: build/Dockerfile
platforms: linux/amd64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

29 changes: 29 additions & 0 deletions .github/workflows/scorecard.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: scorecard

on:
push:
branches:
# Run on pushes to default branch
- main
schedule:
# Run weekly on Saturdays
- cron: "30 1 * * 6"
# Run when branch protection rules change
branch_protection_rule:
# Run the workflow manually
workflow_dispatch:

# Declare default permissions as read-only
permissions: read-all

jobs:
run-scorecard:
# Call reusable workflow file
uses: cisco-ospo/.github/.github/workflows/_scorecard.yml@main
permissions:
id-token: write
security-events: write
secrets: inherit
with:
# Publish results of Scorecard analysis
publish-results: true
71 changes: 71 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@
.DS_Store
.idea

# VS code files
.vscode

# log files
.txt

# Third party
*/third_party/*

# Binaries for programs and plugins
*.exe
*.exe~
Expand Down Expand Up @@ -38,4 +47,66 @@ yapf*.py
*.bak

# fiab related
mlruns
mongodata
postgresdata

# openapi folder generated by openapi.sh
scripts/openapi

# helm dependencies
charts
requirements.lock

# React dependencies
node_modules

# FedScale Dataset
third_party/benchmark/dataset/data/

# Torchvision built-in datasets
lib/python/examples/**/data/

# Object and binary from SPRIGHT
third_party/spright_utility/**/*.o
third_party/spright_utility/**/*.d
third_party/spright_utility/bin/

# ignore dataset and wandb generated files
*/data/*
*wandb
fednlp_data*
cache_dir

scripts/logs/*
scripts/plotters/output/*
scripts/plotters/plots/*
scripts/plotters/old_output/*
scripts/plotters/old_plots/*

# Files on server to ignore
src/*
lib/python/examples/fwdllm/expts/run_tc_expts/log/
requirements_cifar_no_versions.txt
lib/python/examples/fwdllm/expts/run_tc_expts/mpi_host_file
scripts/logs/*
scripts/plotters/output/*
scripts/plotters/plots/*
scripts/plotters/old_output/*
scripts/plotters/old_plots/*

# Local json files (fwdllm) - populated with $USER - do not commit
/tmp_expanded_configs_*
specs/*

# Experiment run outputs (telemetry / plots / logs) — generated, not source
lib/python/examples/async_cifar10/experiments/run_*/
lib/python/examples/async_cifar10/experiments/smoke_*/
lib/python/examples/fwdllm/experiments/
lib/python/examples/fwdllm/expt_scripts/smoke_logs/
# Parity-checker report outputs — generated by scripts/parity_check.py, not source
lib/python/examples/async_cifar10/experiments/parity_*.json
lib/python/examples/async_cifar10/parity_*.json
# Stray trainer log + local Claude Code project dir
temp_trainer.log
.claude/
Loading