\u628a\u4e00\u4e9b\u7570\u5e38\u503c\u5f9e dataset \u4e2d\u6311\u51fa\u4f86<\/p>\n
\u56db\u5f35\u5716\u8868\u8868\u793a\u56db\u7d44\u57fa\u672c\u7684\u7d71\u8a08\u7279\u6027\u4e00\u81f4\u7684\u6578\u64da\uff0c\u4f46\u662f\u5404\u81ea\u756b\u51fa\u4f86\u7684\u5716\u8868\u5b8c\u5168\u4e0d\u540c
\n\u4e3b\u8981\u662f\u5728\u8aaa\u7d71\u8a08\u65b9\u6cd5\u6709\u5176\u4fb7\u9650\u548c\u96e2\u7fa4\u503c\u5c0d\u7d71\u8a08\u7684\u5f71\u97ff\u4e4b\u5927
\n\u9084\u6709\u5c31\u662f\u5206\u6790\u6578\u64da\u524d\u61c9\u8a72\u8981\u5148\u756b\u5716\u8868<\/p>\n
ref: \u627e\u51fa\u8cc7\u6599\u4e4b\u9593\u7684\u96b1\u542b\u95dc\u4fc2 \u662f\u4e00\u7a2e model selection \u7684\u65b9\u6cd5<\/p>\n \u5927\u90e8\u5206\u7684 machine learning \u6a21\u578b\u90fd\u662f\u5728\u8a08\u7b97 cost function ref: cross-validation \u5e38\u5e38\u7528\u4f86\u505a hyperparameter tuning \u7b49\u5230\u6e2c\u8a66\u904e\u6240\u6709\u8d85\u53c3\u6578\u7684\u7d44\u5408\u4e4b\u5f8c ref: \u4e0d\u904e\u5982\u679c\u4f60\u7684 dataset \u771f\u7684\u6c92\u90a3\u9ebc\u5927 ref: \u9084\u6709\u4e00\u7a2e\u65b9\u5f0f\u662f leave-one-out \u6216 leave-n-out \u7576\u6578\u64da\u7684\u7dad\u5ea6\uff08feature \u6578\uff09\u8d85\u904e\u67d0\u500b\u7a0b\u5ea6\u4e4b\u5f8c ref: A smoother boundary corresponds to a simpler model.<\/p>\n \u6bcf\u500b\u7279\u5fb5\u8868\u793a\u70ba\u4e00\u500b\u7dad\u5ea6 \u7b97\u662f unsupervised learning \u7684\u4e00\u7a2e\uff08transformations of the dataset\uff09 ref: \u5f9e\u4e00\u5806 features \u4e2d\u9078\u64c7\u6700\u6709\u7528\u7684 features \u628a\u539f\u672c\u9ad8\u7dad\u5ea6\u7684 features \u8f49\u63db\u6210\u8f03\u5c11\u7dad\u5ea6\u7684 features ref: \u5c31\u662f\u6307\u7d50\u5408\u591a\u7a2e\u6f14\u7b97\u6cd5\u7684 machine learning \u5e38\u898b\u7684 ensemble methods \u6709\uff1a Error = Bias + Variance \u662f\u6307\u6574\u500b\u6a21\u578b\u7684\u6e96\u78ba\u5ea6<\/p>\n Bias \u662f\u6307\u9810\u6e2c\u503c\u548c\u771f\u5be6\u503c\u4e4b\u9593\u7684\u5dee\u8ddd\uff0c\u8868\u793a\u6a21\u578b\u7684\u7cbe\u6e96\u5ea6\uff08\u53cd\u6620\u7684\u662f\u6a21\u578b\u5728\u6a23\u672c\u4e0a\u7684\u8f38\u51fa\u8207\u771f\u5be6\u503c\u4e4b\u9593\u7684\u8aa4\u5dee\uff09 Variance \u662f\u6307\u9810\u6e2c\u503c\u7684\u8b8a\u5316\u7bc4\u570d\uff0c\u8868\u793a\u6a21\u578b\u7684\u7a69\u5b9a\u6027\uff08\u53cd\u6620\u7684\u662f\u6a21\u578b\u6bcf\u4e00\u6b21\u8f38\u51fa\u7d50\u679c\u8207\u6a21\u578b\u8f38\u51fa\u671f\u671b\u4e4b\u9593\u7684\u8aa4\u5dee\uff09 ref: \u5c31\u662f\u627e\u51fa\uff08\u6216\u662f\u5275\u9020\u51fa\uff09\u80fd\u5920\u8b93\u6f14\u7b97\u6cd5\u904b\u4f5c\u5f97\u66f4\u597d\u7684 features \u7684\u904e\u7a0b \u4e00\u6b21\u589e\u52a0\u4e00\u500b feature \u4f86\u8a13\u7df4 model Backwards Stepwise Selection \u5c31\u662f\u53cd\u904e\u4f86<\/p>\n If a model is able to make accurate predictions on unseen data, we say it is able to generalize from the training set to the test set.<\/p>\n \u5c31\u662f\u6307 model \u9810\u6e2c unseen data \u7684\u80fd\u529b ref: \u662f\u4e00\u7a2e\u627e\u51fa\u6700\u5c0f\u7684 cost function \u7684\u6f14\u7b97\u6cd5 \u4e00\u6b21\u53ea\u7528\u4e00\u500b feature \u4f86\u8a13\u7df4 model<\/p>\n In greedy feature selection we choose one feature, train a model and evaluate the performance of the model on a fixed evaluation metric. We keep adding and removing features one-by-one and record performance of the model at every step. We then select the features which have the best evaluation score.<\/p>\n \u5c31\u662f\u5728\u8a13\u7df4 model \u6642\u8f38\u5165\u7684\u53c3\u6578\uff0c\u90a3\u4e9b model \u6c92\u8fa6\u6cd5\u81ea\u5df1\u5b78\u5230\uff0c\u5fc5\u9808\u4eba\u5de5\u6307\u5b9a\u7684\u53c3\u6578\u3002\u901a\u5e38\u6703\u900f\u904e grid search \u548c cross-validation \u7684\u65b9\u5f0f\u9078\u51fa\u6700\u5408\u9069\u7684\u53c3\u6578\u3002<\/p>\n kernel function \u6703\u662f\u4e00\u500b\u8ddd\u96e2\u51fd\u6578<\/p>\n linear kernel \u662f\u6700\u7c21\u55ae\u7684\u4e00\u7a2e kernel function ref: \u7576\u4f60\u6709\u4e00\u5806 data points \u5c31\u662f\u4e00\u689d\u9577\u5f97\u50cf\u982d\u5c3e\u88ab\u62c9\u9577\u62c9\u6241\u7684 S \u7684\u66f2\u7dda<\/p>\n ref: \u91dd\u5c0d\u90a3\u4e9b\u6c92\u6709\u503c\u7684\u6b04\u4f4d\uff0c\u53ef\u80fd\u662f\u7528\u4e2d\u4f4d\u6578\u3001\u5e73\u5747\u503c\u6216\u662f\u6700\u5e38\u898b\u7684\u503c\u4e4b\u985e\u7684\u8cc7\u6599\u586b\u9032\u53bb \u662f\u4e00\u7a2e non-linear dimensionality reduction \u7684\u65b9\u5f0f manifold learning \u901a\u5e38\u7528\u5728 exploratory data analysis ref: \u5c6c\u65bc preprocessing \u7684\u4e00\u90e8\u5206 \u4f8b\u5982\uff1a \u9084\u6709\u53e6\u4e00\u7a2e\u7d71\u8a08\u5b78\u5e38\u7528\u7684\u65b9\u6cd5\uff0c\u662f\u628a\u6578\u503c\u8f49\u63db\u6210 z-scores ref: \u5c31\u662f features<\/p>\n \u4e3b\u6210\u5206\u5206\u6790\uff0c\u662f\u4e00\u79cd\u5206\u6790\u3001\u7b80\u5316\u6570\u636e\u96c6\u7684\u6280\u672f\u3002\u7528\u4e8e\u51cf\u5c11\u6570\u636e\u96c6\u7684\u7ef4\u6570\uff0c\u540c\u65f6\u4fdd\u6301\u6570\u636e\u96c6\u4e2d\u7684\u5bf9\u65b9\u5dee\u8d21\u732e\u6700\u5927\u7684\u7279\u5f81\u3002<\/p>\n \u7528\u4f86 reduce dimensionality\uff08\u6e1b\u5c11 dataset \u7684\u7dad\u5ea6\u6578\uff09 overfitting \u5e38\u5e38\u767c\u751f\u5728 model \u5f88\u8907\u96dc\u3001\u6709\u5f88\u591a\u53c3\u6578\u7684\u6642\u5019 underfitting \u901a\u5e38\u767c\u751f\u5728 model \u592a\u7c21\u55ae\u7684\u6642\u5019 ref: Regularization means explicitly restricting a model to avoid overfitting.<\/p>\n \u662f\u4e00\u7a2e\u9632\u6b62 overfitting \u7684\u6280\u5de7 L1 \u6b63\u5247\u5316\u662f\u6307\u6b0a\u91cd\u5411\u91cf w \u4e2d\u5404\u500b\u5143\u7d20\u7684\u7d55\u5c0d\u503c\u4e4b\u548c<\/p>\n L2 ref: \u5728 classification \u554f\u984c\u4e2d undersampling \u662f\u6307\u6e1b\u5c11\u591a\u6578\u985e\u6a23\u672c\u7684\u6578\u91cf oversampling \u6307\u7684\u662f\u589e\u52a0\u5c11\u6578\u985e\u6a23\u672c\u7684\u6578\u91cf ref: \u628a dataset \u5206\u6210 training set \u548c test set \u4e5f\u53ef\u4ee5\u5206\u6210 training set\u3001validation set\u3001test set \u57fa\u672c\u4e0a\u4f60\u7684 test set \u53ea\u80fd\u7528\u4f86\u8a55\u4f30\u6700\u7d42\u6a21\u578b ref: \u6a5f\u5668\u5b78\u7fd2\u5e38\u898b\u540d\u8a5e\u89e3\u91cb<\/p>\n","protected":false},"author":1,"featured_media":334,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[97],"tags":[98],"class_list":["post-333","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-about-ai","tag-machine-learning"],"_links":{"self":[{"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/posts\/333","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/comments?post=333"}],"version-history":[{"count":0,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/posts\/333\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/media\/334"}],"wp:attachment":[{"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/media?parent=333"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/categories?post=333"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vinta.ws\/code\/wp-json\/wp\/v2\/tags?post=333"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nhttps:\/\/www.wikiwand.com\/zh-tw\/%E5%AE%89%E6%96%AF%E5%BA%93%E5%A7%86%E5%9B%9B%E9%87%8D%E5%A5%8F<\/a><\/p>\nAssociation Rule \u95dc\u806f\u898f\u5247<\/h2>\n
\n\u4f8b\u5982\u77e5\u540d\u7684\u5564\u9152\u8207\u5c3f\u5e03<\/p>\nBest Subset Selection<\/h2>\n
Cost Function \/ Loss Function \u640d\u5931\u51fd\u6578<\/h2>\n
\n\u60f3\u8fa6\u6cd5\u6c42\u51fa\u8b93 cost function \u6700\u5c0f\u5316\u6216\u6700\u5927\u5316\u7684\u5404\u9805\u53c3\u6578
\n\u5e38\u7528\u7684 cost function \u6709 Mean Squared Error (MSE), Root Mean Squared Error (RMSE) \u7b49<\/p>\n
\nhttps:\/\/ml.berkeley.edu\/blog\/2016\/11\/06\/tutorial-1\/<\/a><\/p>\nCross-validation \u4ea4\u53c9\u9a57\u8b49<\/h2>\n
\n\u6700\u4e3b\u6d41\u7684\u65b9\u5f0f\u662f k-fold
\n\u5047\u8a2d k \u662f 3
\n\u4f60\u5148\u628a\u6574\u500b dataset \u62c6\u5206\u6210 training set \u548c test set
\n\u901a\u5e38\u4f60\u6703\u6709\u5f88\u591a\u7d44\u60f3\u8981\u6e2c\u8a66\u7684\u8d85\u53c3\u6578
\n\u5247\u6bcf\u4e00\u7d44\u8d85\u53c3\u6578\u90fd\u6703\u7d93\u6b77\u4ee5\u4e0b\u904e\u7a0b\uff1a<\/p>\n\n
\n\u7528\u8868\u73fe\u6700\u597d\u7684\u90a3\u4e00\u7d44\u8d85\u53c3\u6578\u5c0d\u6574\u500b training set \u518d\u8a13\u7df4\u4e00\u6b21
\n\u5f97\u5230\u6700\u7d42\u7684\u6a21\u578b
\n\u9019\u6642\u5019\u518d\u7528 test set \u4f86\u505a\u6700\u7d42\u6a21\u578b\u7684\u8a55\u4f30<\/p>\n
\nhttps:\/\/spark.apache.org\/docs\/latest\/ml-tuning.html#cross-validation<\/a><\/p>\n
\n\u4e5f\u662f\u53ef\u4ee5\u5c0d\u6574\u500b dataset \u5c0d cross-validation
\n\u5c31\u4e0d\u8981\u5148\u62c6\u5206 training set \u548c test set \u4e86<\/p>\n
\nhttps:\/\/stats.stackexchange.com\/questions\/148688\/cross-validation-with-test-data-set<\/a><\/p>\n
\n\u6bcf\u6b21\u53ea\u7528\u4e00\u500b sample \u4f86\u9a57\u8b49
\n\u5176\u9918\u7684\u90fd\u7528\u4f86\u8a13\u7df4\u6a21\u578b
\n\u76f4\u5230\u6bcf\u500b sample \u90fd\u88ab\u7528\u4f86\u9a57\u8b49\u904e<\/p>\nCurse of Dimensionality \u7dad\u5ea6\u707d\u96e3<\/h2>\n
\n\u5c0e\u81f4\u8a08\u7b97\u7684\u6642\u9593\u904e\u4e45\u3001\u8a18\u61b6\u9ad4\u7528\u91cf\u904e\u5927\uff08\u56e0\u70ba\u662f\u6307\u6578\u578b\u589e\u52a0\uff09
\n\u4e5f\u5fc5\u7136\u6703\u9020\u6210\u6578\u64da\u7a00\u758f
\n\u7279\u5fb5\u8d8a\u591a\u4e5f\u53ef\u80fd\u9020\u6210 overfitting<\/p>\n
\nhttps:\/\/www.quora.com\/What-is-the-curse-of-dimensionality<\/a>
\nhttp:\/\/stats.stackexchange.com\/questions\/169156\/explain-curse-of-dimensionality-to-a-child<\/a><\/p>\nDecision Boundary \u6c7a\u7b56\u908a\u754c<\/h2>\n
\ndecision boundary \u5c31\u662f\u80fd\u5920\u628a\u6574\u500b\u7279\u5fb5\u7a7a\u9593\u88e1\u7684 dataset \u6b63\u78ba\u5283\u5206\u7684\u4e00\u689d\u908a\u754c
\n\u9019\u500b\u908a\u754c\u53ef\u80fd\u662f linear \u6216 non-linear<\/p>\nDimensionality Reduction \u964d\u7dad<\/h2>\n
\n\u53ef\u4ee5\u5206\u6210 feature selection \u548c feature extraction
\n\u5728\u4e0d\u55aa\u5931\u592a\u591a\u8cc7\u8a0a\u7684\u524d\u63d0\u4e0b\u6e1b\u5c11 features \u7684\u7dad\u5ea6
\n\u63db\u500b\u8aaa\u6cd5\u662f\u5617\u8a66\u7528\u66f4\u5c11\u7684\u7dad\u5ea6\u4f86\u8868\u793a\u9019\u500b dataset
\n\u7dad\u5ea6\u6e1b\u5c11\u7684\u597d\u8655\u662f\u63d0\u5347\u8a08\u7b97\u6548\u7387\u548c\u66f4\u5bb9\u6613\u9032\u884c visualization<\/p>\n
\nhttps:\/\/www.wikiwand.com\/en\/Dimensionality_reduction<\/a><\/p>\n
\n\u7a31\u70ba feature selection
\n\u5e38\u898b\u7684\u65b9\u6cd5\u6709 Greedy forward selection<\/p>\n
\n\u7a31\u70ba feature extraction
\n\u8f49\u63db\u4e4b\u5f8c\u5df2\u7d93\u4e0d\u662f\u539f\u672c\u7684\u90a3\u4e9b features \u4e86
\n\u5e38\u898b\u7684\u65b9\u6cd5\u6709 Principal Component Analysis (PCA)\u3001Non-negative Matrix Factorization (NMF)<\/p>\n
\nhttps:\/\/www.wikiwand.com\/en\/Feature_engineering<\/a>
\nhttps:\/\/www.wikiwand.com\/en\/Feature_selection<\/a><\/p>\nEnsemble Learning \u7d44\u5408\u5f0f\u5b78\u7fd2<\/h2>\n
\n\u4f8b\u5982 Random Forest\uff08decision trees + bagging\uff09<\/p>\n
\nbagging (aka bootstrap aggregating)
\nboosting<\/p>\nError \u8aa4\u5dee \/ Bias \u504f\u5dee \/ Variance \u65b9\u5dee<\/h2>\n
\n\u504f\u5dee\u8d8a\u5927\uff0c\u8d8a\u504f\u96e2\u771f\u5be6\u6578\u64da
\n\u56e0\u70ba\u6a21\u578b\u592a\u7c21\u55ae\u800c\u5e36\u4f86\u7684\u9810\u6e2c\u4e0d\u6e96\u78ba >> high bias<\/p>\n
\n\u65b9\u5dee\u8d8a\u5927\uff0c\u6578\u64da\u7684\u5206\u4f48\u8d8a\u5206\u6563
\n\u56e0\u70ba\u6a21\u578b\u592a\u8907\u96dc\u800c\u5e36\u4f86\u7684\u66f4\u5927\u7684\u7a7a\u9593\u8b8a\u5316\u548c\u4e0d\u78ba\u5b9a\u6027 >> high variance<\/p>\n
\nhttps:\/\/www.zhihu.com\/question\/20448464<\/a> \u6709\u5716
\nhttps:\/\/www.zhihu.com\/question\/27068705<\/a><\/p>\nFeature Engineering \u7279\u5fb5\u5de5\u7a0b<\/h2>\n
\n\u4e5f\u53ef\u80fd\u662f\u6574\u5408\u3001\u8f49\u63db\u591a\u500b\u76f8\u95dc\u7684 features \u8b8a\u6210\u4e00\u500b\u65b0\u7684 feature
\n\u901a\u5e38\u6703\u907f\u514d\u4f7f\u7528\u904e\u591a\u7684 features \u9935\u7d66\u6f14\u7b97\u6cd5<\/p>\nForward Stepwise Selection<\/h2>\n
\n\u6bcf\u6b21\u90fd\u8a08\u7b97\u6e96\u78ba\u7387
\n\u76f4\u5230\u6240\u6709 features \u90fd\u7528\u5230<\/p>\nGeneralization \u6cdb\u5316<\/h2>\n
\n\u4f8b\u5982\u4e00\u500b overfitting \u7684 model\uff0c\u5b83\u7684\u6cdb\u5316\u80fd\u529b\u5c31\u4e0d\u597d<\/p>\n
\nhttps:\/\/www.quora.com\/What-is-generalization-in-machine-learning<\/a><\/p>\nGradient Descent \u68af\u5ea6\u4e0b\u964d<\/h2>\n
\n\u4e5f\u5c31\u662f\u627e\u51fa\u6700\u597d\u7684 model parameters<\/p>\nGreedy Feature Selection<\/h2>\n
Hyperparameter \u8d85\u53c3\u6578<\/h2>\n
Kernel Methods<\/h2>\n
\n\u5176\u5be6\u5c31\u662f\u5169\u500b input \u7684 dot product<\/p>\n
\nhttps:\/\/www.zhihu.com\/question\/30371867<\/a><\/p>\nLinear Separability \u7dda\u6027\u53ef\u5206<\/h2>\n
\n\u4f60\u80fd\u5920\u756b\u51fa\u4e00\u689d\u300c\u76f4\u7dda\u300d\u4f86\u5340\u5206\u9019\u4e9b\u9ede\u6642
\n\u5c31\u53ef\u4ee5\u8aaa\u662f linearly separable
\n\u53cd\u800c\u5247\u662f linearly inseparable<\/p>\nLogistic Curve<\/h2>\n
\nhttps:\/\/www.stat.ubc.ca\/~rollin\/teach\/643w04\/lec\/node46.html<\/a><\/p>\nMissing Value Imputation\uff08\u7f3a\u5931\u503c\u586b\u5145\uff09<\/h2>\n
\n\u4e5f\u7a31\u70ba interpolation<\/p>\nManifold Learning<\/h2>\n
\n\u53ef\u4ee5\u7528\u5728\u628a\u9ad8\u7dad\u5ea6\u7684 dataset \u8b8a\u6210\u8f03\u4f4e\u7dad\u5ea6
\n\u4e3b\u8981\u7528\u4f86\u505a visualization
\n\u5e38\u7528\u7684\u6709 t-SNE<\/p>\n
\n\u4e0d\u50cf PCA \u90a3\u6a23\uff0c\u6703\u628a\u7d50\u679c\u7528\u65bc supervised learning \u7684\u8f38\u5165<\/p>\n
\nhttp:\/\/scikit-learn.org\/stable\/modules\/manifold.html<\/a>
\nhttps:\/\/www.wikiwand.com\/en\/Nonlinear_dimensionality_reduction<\/a><\/p>\nNormalization \u6b78\u4e00\u5316\u3001Standarization \u6a19\u6e96\u5316<\/h2>\n
\n\u7d71\u4e00\u5404\u500b\u7279\u5fb5\u7684\u6578\u503c\u7bc4\u570d
\n\u5c0d\u5f88\u591a\u6f14\u7b97\u6cd5\u4f86\u8aaa\u9019\u500b\u6b65\u9a5f\u662f\u5fc5\u8981\u7684<\/p>\n
\n\u7279\u5fb5\u4e00\u662f\u8ddd\u96e2\uff0c\u55ae\u4f4d\u662f\u516c\u5c3a\uff0c\u503c\u7684\u7bc4\u570d\u662f 10 ~ 3000
\n\u7279\u5fb5\u4e8c\u662f\u6a13\u5c64\uff0c\u503c\u7684\u7bc4\u570d\u662f 1 ~ 14
\n\u70ba\u4e86\u907f\u514d\u5c3a\u5ea6\u4e0d\u540c\u9020\u6210\u8aa4\u5c0e
\n\u9700\u8981 rescaling
\n\u628a\u5404\u7a2e\u5c3a\u5ea6\u7684\u6578\u503c\u7d71\u4e00\u8868\u793a\u6210 0 ~ 1 \u4e4b\u9593\u7684\u6578\u5b57
\n\u7a31\u70ba normalization \u6b78\u4e00\u5316<\/p>\n
\n\u4f7f\u6240\u6709\u6578\u64da\u7684\u5e73\u5747\u503c\u70ba 0\u3001\u6a19\u6e96\u5dee\u70ba 1
\n\u7a31\u70ba standarization \u6a19\u6e96\u5316<\/p>\n
\nhttps:\/\/www.quora.com\/What-is-the-difference-between-normalization-standardization-and-regularization-for-data<\/a>
\nhttp:\/\/sobuhu.com\/ml\/2012\/12\/29\/normalization-regularization.html<\/a><\/p>\nPredictors<\/h2>\n
Principal component analysis (PCA) \u4e3b\u6210\u4efd\u5206\u6790<\/h2>\n
\n\u53ef\u4ee5\u627e\u51fa\u5c0d Variance \u8ca2\u737b\u6700\u5927\u7684\u7279\u5fb5<\/p>\nOverfitting \u904e\u5ea6\u64ec\u5408\uff08\u904e\u64ec\u5408\uff09\/ Underfitting \u64ec\u5408\u4e0d\u8db3\uff08\u6b20\u64ec\u5408\uff09<\/h2>\n
\n\u6216\u662f dataset \u88e1\u6709\u5f88\u591a noise \u6216 outlier
\n\u8868\u73fe\u70ba\u5728 training set \u7684\u6e96\u78ba\u7387\u5f88\u9ad8\uff0c\u4f46\u662f\u5728 testing set \u7684\u6e96\u78ba\u7387\u537b\u5f88\u4f4e
\n\u8907\u96dc\u6a21\u578b >> high variance \/ low bias >> overfitting<\/p>\n
\n\u8868\u73fe\u70ba\u5c31\u7b97\u662f\u5728 training set \u4e0a\u7684\u932f\u8aa4\u7387\u5c31\u5f88\u9ad8
\n\u7c21\u55ae\u6a21\u578b >> high bias \/ low variance >> underfitting<\/p>\n
\nhttp:\/\/www.csuldw.com\/2016\/02\/26\/2016-02-26-choosing-a-machine-learning-classifier\/<\/a><\/p>\nRegularization \u6b63\u898f\u5316\u3001\u6b63\u5247\u5316<\/h2>\n
\nregularization \u4fdd\u7559\u6240\u6709 features
\n\u4f46\u662f\u964d\u4f4e\u6216\u61f2\u7f70\u67d0\u4e9b features \u5c0d model \u9810\u6e2c\u503c\u7684\u5f71\u97ff
\n\u5e38\u898b\u7684\u65b9\u6cd5\u6709 L1 \u548c L2<\/p>\n
\n\u6b63\u5247\u5316\u662f\u6307\u6b0a\u91cd\u5411\u91cf w \u4e2d\u5404\u500b\u5143\u7d20\u7684\u5e73\u65b9\u548c\u7136\u5f8c\u518d\u6c42\u5e73\u65b9\u6839<\/p>\n
\nhttps:\/\/zhuanlan.zhihu.com\/p\/25707761<\/a>
\nhttp:\/\/blog.csdn.net\/jinping_shi\/article\/details\/52433975<\/a>
\nhttp:\/\/blog.csdn.net\/zouxy09\/article\/details\/24971995<\/a><\/p>\nResampling<\/h2>\n
\n\u6bcf\u4e00\u7a2e class \u7684\u6578\u91cf\u5dee\u8ddd\u5f88\u5927
\n\u4f8b\u5982\u6b63\u6a23\u672c\u4f54\u4e86 98%\u3001\u8ca0\u6a23\u672c\u4f54\u4e86 2%
\n\u9019\u5c31\u662f\u6240\u8b02\u7684\u4e0d\u5e73\u8861\u7684 dataset
\n\u89e3\u6c7a\u7684\u8fa6\u6cd5\u4e4b\u4e00\u662f resampling
\n\u4e3b\u8981\u53ef\u4ee5\u5206\u6210 oversampling \u548c undersampling\uff08\u904e\u63a1\u6a23\u548c\u6b20\u63a1\u6a23\uff09<\/p>\n
\n\u4f8b\u5982\u96a8\u6a5f\u62ff\u6389\u90e8\u5206\u591a\u6578\u985e\u6a23\u672c
\n\u76f4\u5230\u6b63\u8ca0\u6a23\u672c\u7684\u6578\u91cf\u76f8\u540c
\n\u7f3a\u9ede\u662f\u4f60\u53ef\u80fd\u4e5f\u62ff\u6389\u4e86 dataset \u88e1\u6f5b\u5728\u7684\u8cc7\u8a0a<\/p>\n
\n\u4f8b\u5982\u8907\u88fd\u5c11\u6578\u985e\u6a23\u672c
\n\u8b93\u6b63\u8ca0\u6a23\u672c\u7684\u6578\u91cf\u76e1\u53ef\u80fd\u76f8\u540c
\n\u7f3a\u9ede\u986f\u800c\u6613\u898b\u5c31\u662f\u5bb9\u6613 overfitting
\n\u5176\u4ed6 oversampling \u7684\u65b9\u6cd5\u9084\u6709 SMOTE (Synthetic Minority Over-sampling Technique)
\n\u5408\u6210\u65b0\u7684\u5c11\u6578\u985e\u6a23\u672c
\n\u5408\u6210\u7684\u7b56\u7565\u662f\u5c0d\u6bcf\u500b\u5c11\u6578\u985e\u6a23\u672c a
\n\u5f9e\u5b83\u7684\u6700\u8fd1\u9130\u4e2d\u96a8\u6a5f\u9078\u4e00\u500b\u6a23\u672c b
\n\u7136\u5f8c\u5728 a\u3001b \u4e4b\u9593\u7684\u9023\u7dda\u4e0a\u96a8\u6a5f\u9078\u4e00\u9ede\u4f5c\u70ba\u65b0\u5408\u6210\u7684\u5c11\u6578\u985e\u6a23\u672c<\/p>\n
\nhttp:\/\/www.jiqizhixin.com\/article\/2499<\/a>
\nhttp:\/\/www.algorithmdog.com\/unbalance<\/a>
\nhttps:\/\/en.wikipedia.org\/wiki\/Oversampling_and_undersampling_in_data_analysis<\/a><\/p>\nTraining set \/ Test set<\/h2>\n
\n\u7528 training set \u4f86\u8a13\u7df4\u6a21\u578b
\n\u7528 test set \u4f86\u8a55\u4f30\u7d50\u679c
\n\u9019\u5169\u7d44\u6578\u64da\u5fc5\u9808\u662f\u5f9e\u539f\u59cb\u7684 dataset \u88e1\u300c\u5747\u52fb\u53d6\u6a23\u300d\uff08\u96a8\u6a5f\uff09
\n\u5e38\u898b\u7684\u6bd4\u4f8b\u662f 70\/30
\n\u9019\u7a2e\u65b9\u5f0f\u7a31\u70ba holdout<\/p>\n
\ntraining set \u7528\u4f86\u8a13\u7df4\u6a21\u578b\uff0cvalidation set \u7528\u4f86\u9078\u64c7\u6a21\u578b\uff08\u8abf\u6574\u8d85\u53c3\u6578\uff09\uff0ctesting set \u7528\u5728\u6700\u7d42\u6a21\u578b\u7684\u8a55\u4f30
\n\u5e38\u898b\u7684\u6bd4\u4f8b\u662f 50\/25\/25<\/p>\n
\n\u4e0d\u80fd\u7528 test set \u53bb\u8a13\u7df4\u6a21\u578b\u6216\u662f\u4ea4\u53c9\u9a57\u8b49
\n\u5c0d\u4f60\u7684 model \u4f86\u8aaa test set \u5c31\u662f\u4e00\u7d44 unseen \u7684\u8cc7\u6599
\n\u6240\u4ee5 test set \u7684\u8a55\u4f30\u7d50\u679c\u624d\u53ef\u4ee5\u8996\u70ba model \u4e0a\u7dda\u5f8c\u5c0d\u771f\u5be6\u8cc7\u6599\u7684\u9810\u6e2c\u80fd\u529b
\n\u5982\u679c\u4f60\u7684 model \u5728 validation set \u8868\u73fe\u4e0d\u932f
\n\u4f46\u662f\u5728 test set \u7684\u8868\u73fe\u5f88\u5dee
\n\u90a3\u5c31\u662f overfitting \u4e86<\/p>\n
\nhttps:\/\/stats.stackexchange.com\/questions\/19048\/what-is-the-difference-between-test-set-and-validation-set<\/a>
\nhttps:\/\/www.jiqizhixin.com\/articles\/a62fc871-6366-402b-b32f-f9a3f17a566b<\/a>
\nhttps:\/\/mp.weixin.qq.com\/s\/W7wpxHoC2F5DHCUO7ES1cw<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"