diff --git a/labs/xgboostLab.ipynb b/labs/xgboostLab.ipynb index bd9de85..cd4cdc3 100644 --- a/labs/xgboostLab.ipynb +++ b/labs/xgboostLab.ipynb @@ -16,8 +16,7 @@ { "cell_type": "markdown", "metadata": { - "id": "8L5KyC6KP_yY", - "colab_type": "text" + "id": "8L5KyC6KP_yY" }, "source": [ "# XGBoost Lab\n", @@ -249,7 +248,6 @@ "cell_type": "code", "metadata": { "id": "d3c2iMbCP9aq", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 357 @@ -372,8 +370,7 @@ { "cell_type": "markdown", "metadata": { - "id": "b8hWOON-0qEU", - "colab_type": "text" + "id": "b8hWOON-0qEU" }, "source": [ "And for our first prediction $f_0$ let's predict just the average value:\n" @@ -383,7 +380,6 @@ "cell_type": "code", "metadata": { "id": "ddG9_pVPTwDW", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 357 @@ -512,8 +508,7 @@ { "cell_type": "markdown", "metadata": { - "id": "ijAYdpZd2fzx", - "colab_type": "text" + "id": "ijAYdpZd2fzx" }, "source": [ "and the differences between our predictions and the actual values" @@ -523,7 +518,6 @@ "cell_type": "code", "metadata": { "id": "CKG3OvcB1RWv", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 357 @@ -663,8 +657,7 @@ { "cell_type": "markdown", "metadata": { - "id": "JYYABnmp2niq", - "colab_type": "text" + "id": "JYYABnmp2niq" }, "source": [ "That $y-f_0$ is the residual. What is left, or how far the first classifier was off. The residual is what the second classifier is trying to predict.\n", @@ -678,7 +671,6 @@ "cell_type": "code", "metadata": { "id": "OVqyVkYF2XSO", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 357 @@ -841,8 +833,7 @@ { "cell_type": "markdown", "metadata": { - "id": "T72unS4a7HJP", - "colab_type": "text" + "id": "T72unS4a7HJP" }, "source": [ "And the next classifier will try to predict $y-f_1$ based on x.\n", @@ -867,7 +858,6 @@ "cell_type": "code", "metadata": { "id": "ma21zshP4umI", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 321 @@ -910,8 +900,7 @@ { "cell_type": "markdown", "metadata": { - "id": "U8XREkjWB5fS", - "colab_type": "text" + "id": "U8XREkjWB5fS" }, "source": [ "This reminds me of a section of the *Hitchhiker's Guide to the Galaxy* by Douglas Adams, where Marvin, the robot, is asked to bring two hitchhikers to the bridge and he says:\n", @@ -932,7 +921,6 @@ "cell_type": "code", "metadata": { "id": "83Bp2hvE6bcg", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 360 @@ -975,8 +963,7 @@ { "cell_type": "markdown", "metadata": { - "id": "XZmJUO_2CGez", - "colab_type": "text" + "id": "XZmJUO_2CGez" }, "source": [ "It is a Tesla T4, which has 320 tensor cores.\n", @@ -993,9 +980,7 @@ { "cell_type": "code", "metadata": { - "id": "UwUq3mqwB9EL", - "colab_type": "code", - "colab": {} + "id": "UwUq3mqwB9EL" }, "source": [ "import pandas as pd\n", @@ -1015,8 +1000,7 @@ { "cell_type": "markdown", "metadata": { - "id": "pmHyfTQeCUQs", - "colab_type": "text" + "id": "pmHyfTQeCUQs" }, "source": [ "### Create an instance of the XGBoost classifier\n", @@ -1027,7 +1011,6 @@ "cell_type": "code", "metadata": { "id": "acYGT2PzCRJV", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 154 @@ -1067,8 +1050,7 @@ { "cell_type": "markdown", "metadata": { - "id": "TX_yoP5kDKjl", - "colab_type": "text" + "id": "TX_yoP5kDKjl" }, "source": [ "Let's take a look at those parameters.\n", @@ -1085,7 +1067,6 @@ "cell_type": "code", "metadata": { "id": "MBxXaHTVCbt0", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 154 @@ -1121,8 +1102,7 @@ { "cell_type": "markdown", "metadata": { - "id": "xekQss6bFp65", - "colab_type": "text" + "id": "xekQss6bFp65" }, "source": [ "### evaluate model\n", @@ -1133,7 +1113,6 @@ "cell_type": "code", "metadata": { "id": "SO-caKnjFlG-", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 34 @@ -1165,8 +1144,7 @@ { "cell_type": "markdown", "metadata": { - "id": "hoX-aaylF8He", - "colab_type": "text" + "id": "hoX-aaylF8He" }, "source": [ "We ran a state-of-the-art algorithm on a GPU. Yay us!\n", @@ -1205,7 +1183,6 @@ "cell_type": "code", "metadata": { "id": "6O1qfbyAFvgW", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 554 @@ -1490,8 +1467,7 @@ { "cell_type": "markdown", "metadata": { - "id": "y-Iodm2UOrs7", - "colab_type": "text" + "id": "y-Iodm2UOrs7" }, "source": [ "## divide features and labels\n", @@ -1502,7 +1478,6 @@ "cell_type": "code", "metadata": { "id": "8I01XNJzPOI2", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 417 @@ -1774,8 +1749,7 @@ { "cell_type": "markdown", "metadata": { - "id": "hRTPkOHzNITR", - "colab_type": "text" + "id": "hRTPkOHzNITR" }, "source": [ "Now let's one hot encode the features using sklearn's OneHotEncoder.\n" @@ -1785,7 +1759,6 @@ "cell_type": "code", "metadata": { "id": "JCItEsf8K2TZ", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 51 @@ -1819,8 +1792,7 @@ { "cell_type": "markdown", "metadata": { - "id": "QZyAn3WLZMqV", - "colab_type": "text" + "id": "QZyAn3WLZMqV" }, "source": [ "Fantastic! \n", @@ -1833,7 +1805,6 @@ "cell_type": "code", "metadata": { "id": "pc-AXr9FZGhW", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 51 @@ -1865,8 +1836,7 @@ { "cell_type": "markdown", "metadata": { - "id": "fRDTurCqa9nq", - "colab_type": "text" + "id": "fRDTurCqa9nq" }, "source": [ "You may have noticed that we put a whopping 70% of the data in the test set. We did this because when we are just playing with things to gain an understanding we don't want to wait hours for a result.\n", @@ -1880,9 +1850,7 @@ { "cell_type": "code", "metadata": { - "id": "1Py0OOb5aBy3", - "colab_type": "code", - "colab": {} + "id": "1Py0OOb5aBy3" }, "source": [ "## TO DO call it model\n" @@ -1893,8 +1861,7 @@ { "cell_type": "markdown", "metadata": { - "id": "eks7EiOebgkj", - "colab_type": "text" + "id": "eks7EiOebgkj" }, "source": [ "Now let's say we want to find the best hyperparameter values for \n", @@ -1909,9 +1876,7 @@ { "cell_type": "code", "metadata": { - "id": "JkqwXndrbfhP", - "colab_type": "code", - "colab": {} + "id": "JkqwXndrbfhP" }, "source": [ "# TODO\n", @@ -1923,8 +1888,7 @@ { "cell_type": "markdown", "metadata": { - "id": "b4GoFt4hc4zt", - "colab_type": "text" + "id": "b4GoFt4hc4zt" }, "source": [ "\n", @@ -1937,9 +1901,7 @@ { "cell_type": "code", "metadata": { - "id": "gZkjYTZHc1iP", - "colab_type": "code", - "colab": {} + "id": "gZkjYTZHc1iP" }, "source": [ "# TO DO\n", @@ -1957,8 +1919,7 @@ { "cell_type": "markdown", "metadata": { - "id": "3syNvBizdMYj", - "colab_type": "text" + "id": "3syNvBizdMYj" }, "source": [ "Let's fit the model (this will take awhile)" @@ -1968,7 +1929,6 @@ "cell_type": "code", "metadata": { "id": "BVxaKf8OdK9n", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 157 @@ -2011,8 +1971,7 @@ { "cell_type": "markdown", "metadata": { - "id": "lhJmfE6gM6IJ", - "colab_type": "text" + "id": "lhJmfE6gM6IJ" }, "source": [ "Now let's see what the best parameters are, make predictions on our test data, and check accuracy..." @@ -2022,7 +1981,6 @@ "cell_type": "code", "metadata": { "id": "CFPTUpq_dT0s", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 34 @@ -2051,9 +2009,7 @@ { "cell_type": "code", "metadata": { - "id": "PTboDXG8xsRV", - "colab_type": "code", - "colab": {} + "id": "PTboDXG8xsRV" }, "source": [ "predictions = random_search.best_estimator_.predict(adult_test_features)" @@ -2065,7 +2021,6 @@ "cell_type": "code", "metadata": { "id": "dA0tfa2n7t2w", - "colab_type": "code", "colab": { "base_uri": "https://localhost:8080/", "height": 34 @@ -2094,8 +2049,7 @@ { "cell_type": "markdown", "metadata": { - "id": "zmC50ToeLn7S", - "colab_type": "text" + "id": "zmC50ToeLn7S" }, "source": [ "### This ends our first look at XGBoost\n", @@ -2105,9 +2059,7 @@ { "cell_type": "code", "metadata": { - "id": "UTPwWCmh7y8k", - "colab_type": "code", - "colab": {} + "id": "UTPwWCmh7y8k" }, "source": [ ""