-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathtraining_output.log
More file actions
370 lines (299 loc) · 44.8 KB
/
Copy pathtraining_output.log
File metadata and controls
370 lines (299 loc) · 44.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
python : C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\tracking\_tracking_service\utils.py:177:
FutureWarning: The filesystem tracking backend (e.g., './mlruns') will be deprecated in February 2026. Consider transitioning to a
database backend (e.g., 'sqlite:///mlflow.db') to take advantage of the latest MLflow features. See
https://github.com/mlflow/mlflow/issues/18534 for more details and migration guidance. For migrating existing data,
https://github.com/mlflow/mlflow-export-import can be used.
At line:1 char:1
+ python Scripts/model_training.py 2>&1 | Tee-Object -FilePath "trainin ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (C:\Users\ASUS\A...rt can be used.:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
return FileStore(store_uri, store_uri)
WARNING:root:Malformed experiment '1'. Detailed error Yaml file 'D:\ML 101\ML_101_BootCamp\mlruns\1\meta.yaml' does not exist.
Traceback (most recent call last):
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 379, in
search_experiments
exp = self._get_experiment(exp_id, view_type)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 477, in
_get_experiment
meta = FileStore._read_yaml(experiment_dir, FileStore.META_DATA_FILE_NAME)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 1662, in
_read_yaml
return _read_helper(root, file_name, attempts_remaining=retries)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 1655, in
_read_helper
result = read_yaml(root, file_name)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\utils\yaml_utils.py", line 104, in read_yaml
raise MissingConfigException(f"Yaml file '{file_path}' does not exist.")
mlflow.exceptions.MissingConfigException: Yaml file 'D:\ML 101\ML_101_BootCamp\mlruns\1\meta.yaml' does not exist.
WARNING:root:Malformed experiment 'best_model_artifacts'. Detailed error Yaml file 'D:\ML
101\ML_101_BootCamp\mlruns\best_model_artifacts\meta.yaml' does not exist.
Traceback (most recent call last):
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 379, in
search_experiments
exp = self._get_experiment(exp_id, view_type)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 477, in
_get_experiment
meta = FileStore._read_yaml(experiment_dir, FileStore.META_DATA_FILE_NAME)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 1662, in
_read_yaml
return _read_helper(root, file_name, attempts_remaining=retries)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 1655, in
_read_helper
result = read_yaml(root, file_name)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\utils\yaml_utils.py", line 104, in read_yaml
raise MissingConfigException(f"Yaml file '{file_path}' does not exist.")
mlflow.exceptions.MissingConfigException: Yaml file 'D:\ML 101\ML_101_BootCamp\mlruns\best_model_artifacts\meta.yaml' does not exist.
WARNING:root:Malformed experiment 'production_models'. Detailed error Yaml file 'D:\ML
101\ML_101_BootCamp\mlruns\production_models\meta.yaml' does not exist.
Traceback (most recent call last):
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 379, in
search_experiments
exp = self._get_experiment(exp_id, view_type)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 477, in
_get_experiment
meta = FileStore._read_yaml(experiment_dir, FileStore.META_DATA_FILE_NAME)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 1662, in
_read_yaml
return _read_helper(root, file_name, attempts_remaining=retries)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\store\tracking\file_store.py", line 1655, in
_read_helper
result = read_yaml(root, file_name)
File "C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\utils\yaml_utils.py", line 104, in read_yaml
raise MissingConfigException(f"Yaml file '{file_path}' does not exist.")
mlflow.exceptions.MissingConfigException: Yaml file 'D:\ML 101\ML_101_BootCamp\mlruns\production_models\meta.yaml' does not exist.
fatal: not a git repository (or any of the parent directories): .git
[I 2026-01-21 01:40:15,065] A new study created in memory with name: no-name-aa398431-b5c8-40dc-8713-f7fd4bf1ca62
============================================================
Step 1: Data Preprocessing
============================================================
[OK] Data loaded: 440833 rows, 12 columns
[OK] Preprocessing pipeline built
[OK] Removed 1 rows with missing values
[OK] Preprocessing applied: 440832 rows, 13 features
[OK] Preprocessed data saved: D:\ML 101\ML_101_BootCamp\Data\customer_churn_dataset_prepared.csv
Warning: Could not retrieve git information: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128.
============================================================
Step 2: Loading Dataset
============================================================
Dataset loaded: (440832, 14)
Train: (352665, 13), Test: (88167, 13)
Train class distribution: {1.0: np.int64(199999), 0.0: np.int64(152666)}
Test class distribution: {1.0: np.int64(50000), 0.0: np.int64(38167)}
Train minority/majority ratio: 0.763
Classes are balanced (ratio >= 0.3). SMOTE not needed.
Model 'logistic_regression' registered
Model 'decision_tree' registered
Model 'random_forest' registered
============================================================
Step 3: Training Models with Optuna Optimization
============================================================
Running locally - using 5 trials
[1/3] Training Logistic Regression...
0%| | 0/5 [00:00<?,
?it/s]C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\sklearn\linear_model\_logistic.py:406:
ConvergenceWarning: lbfgs failed to converge after 286 iteration(s) (status=1):
STOP: TOTAL NO. OF ITERATIONS REACHED LIMIT
Increase the number of iterations to improve the convergence (max_iter=286).
You might also want to scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
0%| | 0/5 [00:07<?, ?it/s]
Best trial: 0. Best value: 0.902331: 0%| | 0/5 [00:07<?, ?it/s]
Best trial: 0. Best value: 0.902331: 20%|ΓûêΓûê | 1/5 [00:07<00:29,
7.27s/it]C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\sklearn\linear_model\_logistic.py:406:
ConvergenceWarning: lbfgs failed to converge after 104 iteration(s) (status=1):
STOP: TOTAL NO. OF ITERATIONS REACHED LIMIT
Increase the number of iterations to improve the convergence (max_iter=104).
You might also want to scale the data as shown in:
https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
n_iter_i = _check_optimize_result(
Best trial: 0. Best value: 0.902331: 20%|ΓûêΓûê | 1/5 [00:09<00:29, 7.27s/it]
Best trial: 0. Best value: 0.902331: 20%|ΓûêΓûê | 1/5 [00:09<00:29, 7.27s/it]
Best trial: 0. Best value: 0.902331: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:09<00:13, 4.48s/it]
Best trial: 0. Best value: 0.902331: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:14<00:13, 4.48s/it]
Best trial: 2. Best value: 0.903003: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:14<00:13, 4.48s/it]
Best trial: 2. Best value: 0.903003: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:14<00:09, 4.68s/it]
Best trial: 2. Best value: 0.903003: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:19<00:09, 4.68s/it]
Best trial: 3. Best value: 0.904267: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:19<00:09, 4.68s/it]
Best trial: 3. Best value: 0.904267: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:19<00:04, 4.73s/it]
Best trial: 3. Best value: 0.904267: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:29<00:04, 4.73s/it]
Best trial: 3. Best value: 0.904267: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:29<00:04, 4.73s/it]
Best trial: 3. Best value: 0.904267: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 5/5 [00:29<00:00, 6.54s/it]
Best trial: 3. Best value: 0.904267: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 5/5 [00:29<00:00, 5.85s/it]
2026/01/21 01:41:56 WARNING mlflow.models.model: `artifact_path` is deprecated. Please use `name` instead.
[I 2026-01-21 01:42:06,573] A new study created in memory with name: no-name-8d14c36a-eb0a-4617-9fbb-a9759b81aa50
C:\Users\ASUS\AppData\Local\Programs\Python\Python314\Lib\site-packages\mlflow\tracking\_model_registry\utils.py:215: FutureWarning:
The filesystem model registry backend (e.g., './mlruns') will be deprecated in February 2026. Consider transitioning to a database
backend (e.g., 'sqlite:///mlflow.db') to take advantage of the latest MLflow features. See
https://github.com/mlflow/mlflow/issues/18534 for more details and migration guidance. For migrating existing data,
https://github.com/mlflow/mlflow-export-import can be used.
return FileStore(store_uri)
Registered model 'churn_predictor_lr' already exists. Creating a new version of this model...
Created version '3' of model 'churn_predictor_lr'.
[I 2026-01-21 01:40:22,334] Trial 0 finished with value: 0.9023312758251288 and parameters: {'C': 0.0014132689654478695, 'max_iter': 286, 'solver': 'lbfgs'}. Best is trial 0 with value: 0.9023312758251288.
[I 2026-01-21 01:40:24,862] Trial 1 finished with value: 0.8921683818167085 and parameters: {'C': 0.0014992386689503966, 'max_iter': 104, 'solver': 'lbfgs'}. Best is trial 0 with value: 0.9023312758251288.
[I 2026-01-21 01:40:29,782] Trial 2 finished with value: 0.9030034867361295 and parameters: {'C': 0.002029962766925425, 'max_iter': 245, 'solver': 'newton-cg'}. Best is trial 2 with value: 0.9030034867361295.
[I 2026-01-21 01:40:34,588] Trial 3 finished with value: 0.9042670477746019 and parameters: {'C': 26.075692542891893, 'max_iter': 396, 'solver': 'liblinear'}. Best is trial 3 with value: 0.9042670477746019.
[I 2026-01-21 01:40:44,328] Trial 4 finished with value: 0.9042242593764674 and parameters: {'C': 6.301844997180247, 'max_iter': 361, 'solver': 'newton-cg'}. Best is trial 3 with value: 0.9042670477746019.
Best params: {'C': 26.075692542891893, 'max_iter': 396, 'solver': 'liblinear'}
Best F1 score: 0.9043
============================================================
Performing 5-Fold Stratified Cross-Validation for lr_optuna
============================================================
ACCURACY: 0.8962 (+/- 0.0011)
PRECISION: 0.9399 (+/- 0.0012)
RECALL: 0.8727 (+/- 0.0015)
F1: 0.9051 (+/- 0.0010)
AUC: 0.9595 (+/- 0.0007)
============================================================
==================================================
Model: lr_optuna
==================================================
⚠️ Test Set Performance (may be overly optimistic):
Accuracy: 0.8936
Precision: 0.9235
Recall: 0.8858
F1 Score: 0.9043
AUC: 0.9590
✅ Cross-Validation Performance (more realistic):
F1 Score: 0.9051 (+/- 0.0010)
Accuracy: 0.8962 (+/- 0.0011)
Confusion Matrix (Test Set):
[[35298 2869]
[ 6427 43573]]
[OK] Model 'churn_predictor_lr' v3 registered (Staging)
F1: 0.9043 | Accuracy: 0.8936 | AUC: 0.9590
[2/3] Training Decision Tree...
0%| | 0/5 [00:00<?, ?it/s]
0%| | 0/5 [00:01<?, ?it/s]
Best trial: 0. Best value: 0.99993: 0%| | 0/5 [00:01<?, ?it/s]
Best trial: 0. Best value: 0.99993: 20%|ΓûêΓûê | 1/5 [00:01<00:04, 1.12s/it]
Best trial: 0. Best value: 0.99993: 20%|ΓûêΓûê | 1/5 [00:02<00:04, 1.12s/it]
Best trial: 0. Best value: 0.99993: 20%|ΓûêΓûê | 1/5 [00:02<00:04, 1.12s/it]
Best trial: 0. Best value: 0.99993: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:02<00:03, 1.10s/it]
Best trial: 0. Best value: 0.99993: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:03<00:03, 1.10s/it]
Best trial: 0. Best value: 0.99993: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:03<00:03, 1.10s/it]
Best trial: 0. Best value: 0.99993: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:03<00:02, 1.09s/it]
Best trial: 0. Best value: 0.99993: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:04<00:02, 1.09s/it]
Best trial: 0. Best value: 0.99993: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:04<00:02, 1.09s/it]
Best trial: 0. Best value: 0.99993: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:04<00:01, 1.08s/it]
Best trial: 0. Best value: 0.99993: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:05<00:01, 1.08s/it]
Best trial: 4. Best value: 0.99994: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:05<00:01, 1.08s/it]
Best trial: 4. Best value: 0.99994: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 5/5 [00:05<00:00, 1.10s/it]
Best trial: 4. Best value: 0.99994: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 5/5 [00:05<00:00, 1.10s/it]
2026/01/21 01:43:07 WARNING mlflow.models.model: `artifact_path` is deprecated. Please use `name` instead.
[I 2026-01-21 01:43:13,416] A new study created in memory with name: no-name-1af6a77a-56c2-4d1c-9b8e-6dc035fd0e87
Registered model 'churn_predictor_dt' already exists. Creating a new version of this model...
Created version '3' of model 'churn_predictor_dt'.
[I 2026-01-21 01:42:07,698] Trial 0 finished with value: 0.999929999299993 and parameters: {'max_depth': 15, 'min_samples_split': 16, 'min_samples_leaf': 6, 'criterion': 'entropy'}. Best is trial 0 with value: 0.999929999299993.
[I 2026-01-21 01:42:08,775] Trial 1 finished with value: 0.999919998399968 and parameters: {'max_depth': 19, 'min_samples_split': 17, 'min_samples_leaf': 4, 'criterion': 'gini'}. Best is trial 0 with value: 0.999929999299993.
[I 2026-01-21 01:42:09,846] Trial 2 finished with value: 0.9997299432880905 and parameters: {'max_depth': 13, 'min_samples_split': 18, 'min_samples_leaf': 1, 'criterion': 'gini'}. Best is trial 0 with value: 0.999929999299993.
[I 2026-01-21 01:42:10,930] Trial 3 finished with value: 0.9993295910505409 and parameters: {'max_depth': 12, 'min_samples_split': 13, 'min_samples_leaf': 6, 'criterion': 'gini'}. Best is trial 0 with value: 0.999929999299993.
[I 2026-01-21 01:42:12,058] Trial 4 finished with value: 0.99994 and parameters: {'max_depth': 20, 'min_samples_split': 13, 'min_samples_leaf': 1, 'criterion': 'entropy'}. Best is trial 4 with value: 0.99994.
Best params: {'max_depth': 20, 'min_samples_split': 13, 'min_samples_leaf': 1, 'criterion': 'entropy'}
Best F1 score: 0.9999
============================================================
Performing 5-Fold Stratified Cross-Validation for dt_optuna
============================================================
ACCURACY: 0.9999 (+/- 0.0000)
PRECISION: 1.0000 (+/- 0.0000)
RECALL: 0.9999 (+/- 0.0000)
F1: 0.9999 (+/- 0.0000)
AUC: 1.0000 (+/- 0.0000)
============================================================
==================================================
Model: dt_optuna
==================================================
⚠️ Test Set Performance (may be overly optimistic):
Accuracy: 0.9999
Precision: 0.9999
Recall: 0.9999
F1 Score: 0.9999
AUC: 0.9999
✅ Cross-Validation Performance (more realistic):
F1 Score: 0.9999 (+/- 0.0000)
Accuracy: 0.9999 (+/- 0.0000)
Confusion Matrix (Test Set):
[[38167 0]
[ 3 49997]]
[OK] Model 'churn_predictor_dt' v3 registered (Staging)
F1: 0.9999 | Accuracy: 0.9999 | AUC: 0.9999
[3/3] Training Random Forest...
0%| | 0/5 [00:00<?, ?it/s]
0%| | 0/5 [00:13<?, ?it/s]
Best trial: 0. Best value: 0.993731: 0%| | 0/5 [00:13<?, ?it/s]
Best trial: 0. Best value: 0.993731: 20%|ΓûêΓûê | 1/5 [00:13<00:55, 13.82s/it]
Best trial: 0. Best value: 0.993731: 20%|ΓûêΓûê | 1/5 [00:21<00:55, 13.82s/it]
Best trial: 0. Best value: 0.993731: 20%|ΓûêΓûê | 1/5 [00:21<00:55, 13.82s/it]
Best trial: 0. Best value: 0.993731: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:21<00:30, 10.21s/it]
Best trial: 0. Best value: 0.993731: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:30<00:30, 10.21s/it]
Best trial: 0. Best value: 0.993731: 40%|ΓûêΓûêΓûêΓûê | 2/5 [00:30<00:30, 10.21s/it]
Best trial: 0. Best value: 0.993731: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:30<00:19, 9.73s/it]
Best trial: 0. Best value: 0.993731: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:36<00:19, 9.73s/it]
Best trial: 0. Best value: 0.993731: 60%|ΓûêΓûêΓûêΓûêΓûêΓûê | 3/5 [00:36<00:19, 9.73s/it]
Best trial: 0. Best value: 0.993731: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:36<00:08, 8.12s/it]
Best trial: 0. Best value: 0.993731: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:43<00:08, 8.12s/it]
Best trial: 0. Best value: 0.993731: 80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 4/5 [00:43<00:08, 8.12s/it]
Best trial: 0. Best value: 0.993731: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 5/5 [00:43<00:00, 7.69s/it]
Best trial: 0. Best value: 0.993731: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 5/5 [00:43<00:00, 8.65s/it]
2026/01/21 01:46:37 WARNING mlflow.models.model: `artifact_path` is deprecated. Please use `name` instead.
Registered model 'churn_predictor_rf' already exists. Creating a new version of this model...
Created version '2' of model 'churn_predictor_rf'.
[I 2026-01-21 01:43:27,237] Trial 0 finished with value: 0.9937313222583339 and parameters: {'n_estimators': 55, 'max_depth': 12, 'min_samples_split': 14, 'min_samples_leaf': 5, 'criterion': 'entropy'}. Best is trial 0 with value: 0.9937313222583339.
[I 2026-01-21 01:43:34,922] Trial 1 finished with value: 0.9908144025328454 and parameters: {'n_estimators': 37, 'max_depth': 9, 'min_samples_split': 19, 'min_samples_leaf': 10, 'criterion': 'gini'}. Best is trial 0 with value: 0.9937313222583339.
[I 2026-01-21 01:43:44,070] Trial 2 finished with value: 0.9900840437753189 and parameters: {'n_estimators': 47, 'max_depth': 8, 'min_samples_split': 17, 'min_samples_leaf': 6, 'criterion': 'entropy'}. Best is trial 0 with value: 0.9937313222583339.
[I 2026-01-21 01:43:49,727] Trial 3 finished with value: 0.9895366519300295 and parameters: {'n_estimators': 24, 'max_depth': 9, 'min_samples_split': 16, 'min_samples_leaf': 6, 'criterion': 'gini'}. Best is trial 0 with value: 0.9937313222583339.
[I 2026-01-21 01:43:56,664] Trial 4 finished with value: 0.9905531974956395 and parameters: {'n_estimators': 23, 'max_depth': 8, 'min_samples_split': 16, 'min_samples_leaf': 10, 'criterion': 'entropy'}. Best is trial 0 with value: 0.9937313222583339.
Best params: {'n_estimators': 55, 'max_depth': 12, 'min_samples_split': 14, 'min_samples_leaf': 5, 'criterion': 'entropy'}
Best F1 score: 0.9937
============================================================
Performing 5-Fold Stratified Cross-Validation for rf_optuna
============================================================
ACCURACY: 0.9926 (+/- 0.0004)
PRECISION: 1.0000 (+/- 0.0000)
RECALL: 0.9870 (+/- 0.0007)
F1: 0.9934 (+/- 0.0003)
AUC: 1.0000 (+/- 0.0000)
============================================================
==================================================
Model: rf_optuna
==================================================
⚠️ Test Set Performance (may be overly optimistic):
Accuracy: 0.9931
Precision: 0.9999
Recall: 0.9879
F1 Score: 0.9939
AUC: 1.0000
✅ Cross-Validation Performance (more realistic):
F1 Score: 0.9934 (+/- 0.0003)
Accuracy: 0.9926 (+/- 0.0004)
Confusion Matrix (Test Set):
[[38164 3]
[ 558 49442]]
[OK] Model 'churn_predictor_rf' v2 registered (Staging)
F1: 0.9939 | Accuracy: 0.9931 | AUC: 1.0000
============================================================
Step 4: Model Performance Comparison
============================================================
model f1 accuracy precision recall auc
decision_tree 0.999940 0.999932 0.999940 0.99994 0.999944
random_forest 0.993863 0.993081 0.999939 0.98786 0.999992
logistic_regression 0.904267 0.893634 0.923499 0.88582 0.959019
============================================================
Step 5: Promoting Best Model to Production
============================================================
--- Promotion Analysis (Metric: f1_score) ---
Current Champion: churn_predictor_dt v2 (Score: 1.0000)
Best Challenger: churn_predictor_dt v2 (Score: 1.0000)
-> Current Production model remains superior. No promotion performed.
============================================================
[OK] Training Pipeline Complete!
============================================================
MLflow UI: file:///D:/ML 101/ML_101_BootCamp/mlruns
Production models: D:\ML 101\ML_101_BootCamp\mlruns\production_models
============================================================