{"id":1683,"date":"2021-06-13T12:04:43","date_gmt":"2021-06-13T04:04:43","guid":{"rendered":"https:\/\/ykyi.net\/?p=1683"},"modified":"2021-06-13T12:04:43","modified_gmt":"2021-06-13T04:04:43","slug":"%e7%94%a8%e7%90%b4%e7%94%9fjensen%e4%b8%8d%e7%ad%89%e5%bc%8f%e8%af%81%e6%98%8e%e6%9c%9f%e6%9c%9b%e6%9c%80%e5%a4%a7%e5%80%bcem%e7%ae%97%e6%b3%95%e6%94%b6%e6%95%9b","status":"publish","type":"post","link":"https:\/\/ykyi.net\/?p=1683","title":{"rendered":"\u7528\u7434\u751f(Jensen)\u4e0d\u7b49\u5f0f\u8bc1\u660e\u671f\u671b\u6700\u5927\u503c(EM)\u7b97\u6cd5\u6536\u655b"},"content":{"rendered":"

\u4eca\u5e74\u4e8c\u6708\u4efd\u5237\u4e86\u4e00\u904d\u201c\u673a\u5668\u5b66\u4e60\u767d\u677f\u63a8\u5bfc\u201d\u7cfb\u5217\u89c6\u9891\u3002https:\/\/www.bilibili.com\/video\/BV1aE411o7qd \u3002\u8fd9\u4e2a\u673a\u5668\u5b66\u4e60\u7684\u516c\u5f0f\u63a8\u5bfc\u7cfb\u5217\u8bb2\u5730\u771f\u662f\u592a\u597d\u4e86\uff0c\u5f53\u65f6\u50cf\u8ffd\u5267\u4e00\u822c\u82b1\u4e86\u4e24\u4e2a\u661f\u671f\u542c\u5b8c\u3002\u975e\u5e38\u611f\u8c22\u5f55\u8fd9\u4e2a\u89c6\u9891\u7cfb\u5217\u7684\u8001\u5e08shuhuai008\u3002\u4e0d\u8fc7\uff0c\u56e0\u4e3a\u4ee5\u524d\u673a\u5668\u5b66\u4e60\u7684\u57fa\u7840\u51e0\u4e4e\u4e3a0\uff0c\u53c8\u6ca1\u6709\u52a8\u624b\u505a\u76f8\u5173\u7ec3\u4e60\u3002\u4e09\u4e2a\u534a\u6708\u4ee5\u540e\uff0c\u53c8\u5fd8\u4e86\u5f88\u591a\u4e86\u3002\u6628\u5929\uff0c\u6211\u60f3\u8d77\u8981\u590d\u4e60\u4e00\u4e0bEM\u7b97\u6cd5\uff0c\u4e8e\u662f\u53c8\u628a\u89c6\u9891\u6361\u8d77\u6765\u627e\u5230EM\u7b97\u6cd5\u7684\u90e8\u5206\uff0c\u590d\u4e60\u4e00\u4e0b\u3002<\/p>\n

\u7b2c\u4e8c\u6b21\u770b\uff0c\u679c\u7136\u5feb\u4e86\u5f88\u591a\u3002\u4e0d\u8fc7\uff0c\u5728\u7528\u7434\u751f\u4e0d\u7b49\u5f0f\u8bc1\u660eEM\u7b97\u6cd5\u6536\u655b\u7684\u90e8\u5206\uff0c\u5361\u4f4f\u4e86\u3002\u6211\u4e0d\u65ad\u56de\u5fc6\u7b2c\u4e00\u6b21\u770b\u7684\u65f6\u5019\u662f\u5982\u4f55\u7406\u89e3\u7684\uff0c\u5948\u4f55\u56de\u5fc6\u4e0d\u8d77\u6765\u4e86\u3002\u4e8e\u662f\uff0c\u627e\u5230Jensen\u4e0d\u7b49\u5f0f\u7684\u8bcd\u6761\u770b\u4e86\u770b\uff0c\u7422\u78e8\u4e86\u597d\u4e45\u3002\u7ec8\u4e8e\u61c2\u4e86\uff0c\u5728\u8fd9\u91cc\u628a\u8fd9\u4e2a\u601d\u8003\u8fc7\u7a0b\u8bb0\u4e0b\u6765\u3002<\/p>\n

\u6211\u5361\u5728\u8fd9\u4e2a\u6b65\u9aa4\uff1a\u9700\u8981\u7528Jensen Inequality\u8bc1\u660e\u5982\u4e0b\u4e0d\u7b49\u5f0f\uff1a
\n
\n\u4e0a\u9762\u516c\u5f0f\u4e2d\u7684z\u8868\u793a\u6a21\u578b\u4e2d\u7684\u9690\u53d8\u91cf\uff0cx\u8868\u793a\u89c2\u6d4b\u503c\uff0c\\theta<\/span>\u662f\u53c2\u6570\uff0c\\theta^{(t)}<\/span>\u8868\u793atheta\u5728\u7b2ct\u65f6\u523b\u7684\u503c\uff08EM\u7b97\u6cd5\u662f\u4e00\u4e2a\u8fed\u4ee3\u7b97\u6cd5\uff09\u3002<\/p>\n

\u8bc1\u660e\u8fd9\u4e2a\u4e0d\u7b49\u5f0f\u9700\u8981\u7528\u5230\u7434\u751f\u4e0d\u7b49\u5f0f\u5728\u51f9\u51fd\u6570\u4e0a\u7684\u6027\u8d28\u3002\u5982\u4e0b\uff1a
\n\u8bbef(x)\u4e2d\u4e00\u4e2a\u51f9\u51fd\u6570\uff08\u51f9\u51fd\u6570\u56fe\u5f62\u548c\u4e2d\u6587\u5b57\u51f9\u7684\u5f62\u72b6\u76f8\u53cd\uff0c\u53ef\u4ee5\u7406\u89e3\u4e3a\u4e0a\u51f8\uff0clog\u51fd\u6570\u5373\u4e00\u4e2a\u51f9\u51fd\u6570\uff09,\u4e0b\u9762\u7684\u4e0d\u7b49\u5f0f\u6210\u7acb\u3002
\na_1f(x_1) + a_2f(x_2) + \\dots + a_nf(x_n) \\le f(a_1x_1 + a_2x_2 + \\dots + a_nx_n)
\n\u4e14 a_1 + a_2 + \\dots + a_n = 1<\/span>
\n\u7528\u81ea\u7136\u8bed\u8a00\u63cf\u8ff0\u662f\uff1a\u51f9\u51fd\u6570\u5b9a\u4e49\u57df\u5185\u7684n\u4e2a\u81ea\u53d8\u91cf\u5bf9\u5e94\u7684\u56e0\u53d8\u91cf\u7684\u7ebf\u6027\u7ec4\u5408\u5c0f\u4e8e\u6216\u7b49\u4e8en\u4e2a\u81ea\u53d8\u91cf\u5148\u505a\u7ebf\u6027\u7ec4\u5408\u518d\u6c42\u5bf9\u5e94\u7684\u56e0\u53d8\u91cf\uff0c\u4f5c\u7ebf\u6027\u7ec4\u5408\u7684\u7cfb\u6570\u4e4b\u548c\u4e3a1\u3002\u7b80\u5355\u5730\u8bf4\uff1a\u5148\u6c42\u51fd\u6570\u503c\u518d\u505a\u7ebf\u6027\u7ec4\u5408 \u5c0f\u4e8e\u6216\u7b49\u4e8e \u5148\u505a\u7ebf\u6027\u7ec4\u5408\u518d\u6c42\u51fd\u6570\u503c\u3002<\/p>\n

\u518d\u56de\u5230\u6211\u4eec\u8981\u8bc1\u660e\u7684\u4e0d\u7b49\u5f0f\u3002\u79ef\u5206\u53f7\u5373\u53ef\u770b\u6210\u662f\u65e0\u7a77\u591a\u4e2a\u503c\u7684\u7ebf\u6027\u7ec4\u5408\uff0c\u5bf9\u6570\u7b26\u53f7log\u540e\u9762\u5bf9\u5e94\u7684\u90a3\u4e00\u5768
\n{\\frac{P(z|x, \\theta^{(t+1)})} {P(z|x, \\theta^{(t)})} }<\/span>
\n\u770b\u6210\u662flog\u51fd\u6570\u7684\u56e0\u53d8\u91cf\uff0clog\u524d\u9762\u7684
\nP(z|x, \\theta^{(t)})<\/span>
\n\u770b\u6210\u662f\u7ebf\u6027\u7ec4\u5408\u7684\u7cfb\u6570\uff0c\u5f88\u660e\u663e\u8fd9\u4e2a\u7cfb\u6570\u662f\u6ee1\u8db3\u79ef\u5206\u548c\u4e3a1\u7684\u6982\u7387\u5bc6\u5ea6\u3002\u4e8e\u662f\uff0c\u5c31\u53ef\u4ee5\u5957\u7528\u4e0a\u9762\u63d0\u5230\u7684\u51f9\u51fd\u6570\u4e0a\u7684\u7434\u751f\u4e0d\u7b49\u5f0f\u3002\u4e0d\u7b49\u5f0f\u7684\u53f3\u8fb9\u4e3a0\uff0c\u5de6\u8fb9\u662f\u201c\u5148\u6c42\u51fd\u6570\u503c\u518d\u505a\u7ebf\u6027\u7ec4\u5408\u201d\uff0c\u5316\u7b80\u4e3a\u5de6\u8fb9\u5c0f\u4e8e\u201c\u5148\u505a\u7ebf\u6027\u7ec4\u5408\u518d\u6c42\u51fd\u6570\u503c\u201d\uff1a
\n\\begin{aligned}
\n\u5de6\u8fb9 &\\le log_{}{\\int\\limits_z {\\frac{P(z|x, \\theta^{(t+1)})} {P(z|x, \\theta^{(t)})} } P(z|x, \\theta^{(t)})}dz \&=log_{}{}\\int \\limits_zP(z|x,\\theta^{(t+1)})dz=log_{}{1}=0
\n\\end{aligned}<\/span>
\n\u5f97\u8bc1\uff01<\/p>\n","protected":false},"excerpt":{"rendered":"

\u4eca\u5e74\u4e8c\u6708\u4efd\u5237\u4e86\u4e00\u904d\u201c\u673a\u5668\u5b66\u4e60\u767d\u677f\u63a8\u5bfc\u201d\u7cfb\u5217\u89c6\u9891\u3002https:\/\/www.bilibili.com\/video\/BV1aE411o7qd \u3002\u8fd9\u4e2a\u673a\u5668\u5b66\u4e60\u7684\u516c\u5f0f\u63a8\u5bfc\u7cfb\u5217\u8bb2\u5730\u771f\u662f\u592a\u597d\u4e86\uff0c\u5f53\u65f6\u50cf\u8ffd\u5267\u4e00\u822c\u82b1\u4e86\u4e24\u4e2a\u661f\u671f\u542c\u5b8c\u3002\u975e\u5e38\u611f\u8c22\u5f55\u8fd9\u4e2a\u89c6\u9891\u7cfb\u5217\u7684\u8001\u5e08shuhuai008\u3002\u4e0d\u8fc7\uff0c\u56e0\u4e3a\u4ee5\u524d\u673a\u5668\u5b66\u4e60\u7684\u57fa\u7840\u51e0\u4e4e\u4e3a0\uff0c\u53c8\u6ca1\u6709\u52a8\u624b\u505a\u76f8\u5173\u7ec3\u4e60\u3002\u4e09\u4e2a\u534a\u6708\u4ee5\u540e\uff0c\u53c8\u5fd8\u4e86\u5f88\u591a\u4e86\u3002\u6628\u5929\uff0c\u6211\u60f3\u8d77\u8981\u590d\u4e60\u4e00\u4e0bEM\u7b97\u6cd5\uff0c\u4e8e\u662f\u53c8\u628a\u89c6\u9891\u6361\u8d77\u6765\u627e\u5230EM\u7b97\u6cd5\u7684\u90e8\u5206\uff0c\u590d\u4e60\u4e00\u4e0b\u3002 \u7b2c\u4e8c\u6b21\u770b\uff0c\u679c\u7136\u5feb\u4e86\u5f88\u591a\u3002\u4e0d\u8fc7\uff0c\u5728\u7528\u7434\u751f\u4e0d\u7b49\u5f0f\u8bc1\u660eEM\u7b97\u6cd5\u6536\u655b\u7684\u90e8\u5206\uff0c\u5361\u4f4f\u4e86\u3002\u6211\u4e0d\u65ad\u56de\u5fc6\u7b2c\u4e00\u6b21\u770b\u7684\u65f6\u5019\u662f\u5982\u4f55\u7406\u89e3\u7684\uff0c\u5948\u4f55\u56de\u5fc6\u4e0d\u8d77\u6765\u4e86\u3002\u4e8e\u662f\uff0c\u627e\u5230Jensen\u4e0d\u7b49\u5f0f\u7684\u8bcd\u6761\u770b\u4e86\u770b\uff0c\u7422\u78e8\u4e86\u597d\u4e45\u3002\u7ec8\u4e8e\u61c2\u4e86\uff0c\u5728\u8fd9\u91cc\u628a\u8fd9\u4e2a\u601d\u8003\u8fc7\u7a0b\u8bb0\u4e0b\u6765\u3002 \u6211\u5361\u5728\u8fd9\u4e2a\u6b65\u9aa4\uff1a\u9700\u8981\u7528Jensen Inequality\u8bc1\u660e\u5982\u4e0b\u4e0d\u7b49\u5f0f\uff1a \u4e0a\u9762\u516c\u5f0f\u4e2d\u7684z\u8868\u793a\u6a21\u578b\u4e2d\u7684\u9690\u53d8\u91cf\uff0cx\u8868\u793a\u89c2\u6d4b\u503c\uff0c\\theta\u662f\u53c2\u6570\uff0c\\theta^{(t)}\u8868\u793atheta\u5728\u7b2ct\u65f6\u523b\u7684\u503c\uff08EM\u7b97\u6cd5\u662f\u4e00\u4e2a\u8fed\u4ee3\u7b97\u6cd5\uff09\u3002 \u8bc1\u660e\u8fd9\u4e2a\u4e0d\u7b49\u5f0f\u9700\u8981\u7528\u5230\u7434\u751f\u4e0d\u7b49\u5f0f\u5728\u51f9\u51fd\u6570\u4e0a\u7684\u6027\u8d28\u3002\u5982\u4e0b\uff1a \u8bbef(x)\u4e2d\u4e00\u4e2a\u51f9\u51fd\u6570\uff08\u51f9\u51fd\u6570\u56fe\u5f62\u548c\u4e2d\u6587\u5b57\u51f9\u7684\u5f62\u72b6\u76f8\u53cd\uff0c\u53ef\u4ee5\u7406\u89e3\u4e3a\u4e0a\u51f8\uff0clog\u51fd\u6570\u5373\u4e00\u4e2a\u51f9\u51fd\u6570\uff09,\u4e0b\u9762\u7684\u4e0d\u7b49\u5f0f\u6210\u7acb\u3002 a_1f(x_1) + a_2f(x_2) + \\dots + a_nf(x_n) \\le f(a_1x_1 + a_2x_2 + \\dots + a_nx_n) \u4e14 a_1 + a_2 + \\dots + a_n = 1 \u7528\u81ea\u7136\u8bed\u8a00\u63cf\u8ff0\u662f\uff1a\u51f9\u51fd\u6570\u5b9a\u4e49\u57df\u5185\u7684n\u4e2a\u81ea\u53d8\u91cf\u5bf9\u5e94\u7684\u56e0\u53d8\u91cf\u7684\u7ebf\u6027\u7ec4\u5408\u5c0f\u4e8e\u6216\u7b49\u4e8en\u4e2a\u81ea\u53d8\u91cf\u5148\u505a\u7ebf\u6027\u7ec4\u5408\u518d\u6c42\u5bf9\u5e94\u7684\u56e0\u53d8\u91cf\uff0c\u4f5c\u7ebf\u6027\u7ec4\u5408\u7684\u7cfb\u6570\u4e4b\u548c\u4e3a1\u3002\u7b80\u5355\u5730\u8bf4\uff1a\u5148\u6c42\u51fd\u6570\u503c\u518d\u505a\u7ebf\u6027\u7ec4\u5408 \u5c0f\u4e8e\u6216\u7b49\u4e8e \u5148\u505a\u7ebf\u6027\u7ec4\u5408\u518d\u6c42\u51fd\u6570\u503c\u3002 \u518d\u56de\u5230\u6211\u4eec\u8981\u8bc1\u660e\u7684\u4e0d\u7b49\u5f0f\u3002\u79ef\u5206\u53f7\u5373\u53ef\u770b\u6210\u662f\u65e0\u7a77\u591a\u4e2a\u503c\u7684\u7ebf\u6027\u7ec4\u5408\uff0c\u5bf9\u6570\u7b26\u53f7log\u540e\u9762\u5bf9\u5e94\u7684\u90a3\u4e00\u5768 {\\frac{P(z|x, \\theta^{(t+1)})} {P(z|x, \\theta^{(t)})} } \u770b\u6210\u662flog\u51fd\u6570\u7684\u56e0\u53d8\u91cf\uff0clog\u524d\u9762\u7684 P(z|x, \\theta^{(t)}) \u770b\u6210\u662f\u7ebf\u6027\u7ec4\u5408\u7684\u7cfb\u6570\uff0c\u5f88\u660e\u663e\u8fd9\u4e2a\u7cfb\u6570\u662f\u6ee1\u8db3\u79ef\u5206\u548c\u4e3a1\u7684\u6982\u7387\u5bc6\u5ea6\u3002\u4e8e\u662f\uff0c\u5c31\u53ef\u4ee5\u5957\u7528\u4e0a\u9762\u63d0\u5230\u7684\u51f9\u51fd\u6570\u4e0a\u7684\u7434\u751f\u4e0d\u7b49\u5f0f\u3002\u4e0d\u7b49\u5f0f\u7684\u53f3\u8fb9\u4e3a0\uff0c\u5de6\u8fb9\u662f\u201c\u5148\u6c42\u51fd\u6570\u503c\u518d\u505a\u7ebf\u6027\u7ec4\u5408\u201d\uff0c\u5316\u7b80\u4e3a\u5de6\u8fb9\u5c0f\u4e8e\u201c\u5148\u505a\u7ebf\u6027\u7ec4\u5408\u518d\u6c42\u51fd\u6570\u503c\u201d\uff1a \\begin{aligned} \u5de6\u8fb9 &\\le log_{}{\\int\\limits_z {\\frac{P(z|x, \\theta^{(t+1)})} {P(z|x, \\theta^{(t)})} } … Continue reading “\u7528\u7434\u751f(Jensen)\u4e0d\u7b49\u5f0f\u8bc1\u660e\u671f\u671b\u6700\u5927\u503c(EM)\u7b97\u6cd5\u6536\u655b”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,6],"tags":[104],"class_list":["post-1683","post","type-post","status-publish","format-standard","hentry","category-ai","category-tech_articles","tag-104"],"_links":{"self":[{"href":"https:\/\/ykyi.net\/index.php?rest_route=\/wp\/v2\/posts\/1683","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ykyi.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ykyi.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ykyi.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ykyi.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1683"}],"version-history":[{"count":0,"href":"https:\/\/ykyi.net\/index.php?rest_route=\/wp\/v2\/posts\/1683\/revisions"}],"wp:attachment":[{"href":"https:\/\/ykyi.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1683"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ykyi.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1683"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ykyi.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1683"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}