青青草原综合久久大伊人导航_色综合久久天天综合_日日噜噜夜夜狠狠久久丁香五月_热久久这里只有精品

O(1) 的小樂

Job Hunting

公告

記錄我的生活和工作。。。
<2010年12月>
2829301234
567891011
12131415161718
19202122232425
2627282930311
2345678

統計

  • 隨筆 - 182
  • 文章 - 1
  • 評論 - 41
  • 引用 - 0

留言簿(10)

隨筆分類(70)

隨筆檔案(182)

文章檔案(1)

如影隨形

搜索

  •  

最新隨筆

最新評論

閱讀排行榜

評論排行榜

Expectation-maximization algorithm EM算法

     In statistics, an expectation-maximization (EM) algorithm is a method for finding maximum likelihood ormaximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. EM is an iterative method which alternates between performing an expectation (E) step, which computes the expectation of the log-likelihood evaluated using the current estimate for the latent variables, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.

 

  EM算法可用于很多問題的框架,其中需要估計一組描述概率分布的參數\boldsymbol\theta,只給定了由此產生的全部數據中能觀察到的一部分!

  EM算法是一種迭代算法,它由基本的兩個步驟組成:

  E step:估計期望步驟

  使用對隱變量的現有估計來計算log極大似然

  M step: 最大化期望步驟

  計算一個對隱變量更好的估計,使其最大化log似然函數對隱變量Y的期望。用新計算的隱變量參數代替之前的對隱變量的估計,進行下一步的迭代!

 

 

觀測數據:觀測到的隨機變量X的IID樣本:

image

缺失數據:未觀測到的隱含變量(隱變量)Y的值:

image

完整數據: 包含觀測到的隨機變量X和未觀測到的隨機變量Y的數據,Z=(X,Y)

 

似然函數:(似然函數的幾種寫法)

JL})D_HBNI489~H}GCRMWVJ

log似然函數為:

image

E step:用對隱變量的現有估計\boldsymbol\theta^{(t)}計算隱變量Y的期望

  image

其中需要用到貝葉斯公式:

image 

M step:最大化期望,獲得對隱變量更好的估計

image

 

維基中的表述是這樣子:

Given a statistical model consisting of a set \mathbf{X} of observed data, a set of unobserved latent data or missing values Y, and a vector of unknown parameters \boldsymbol\theta, along with a likelihood function L(\boldsymbol\theta; \mathbf{X}, \mathbf{Z}) = p(\mathbf{X}, \mathbf{Z}|\boldsymbol\theta), the maximum likelihood estimate (MLE) of the unknown parameters is determined by the marginal likelihood of the observed data 

       CR%M2I[QD88[N5$3(H))%ZR

However, this quantity is often intractable.

The EM algorithm seeks to find the MLE of the marginal likelihood by iteratively applying the following two steps:

Expectation step (E-step): Calculate the expected value of the log likelihood function, with respect to the conditional distribution of Y given \mathbf{X} under the current estimate of the parameters \boldsymbol\theta^{(t)}:

       A7DFNWMY)KAI]T5)_OMKRUD

Maximization step (M-step): Find the parameter that maximizes this quantity:
\boldsymbol\theta^{(t+1)} = \underset{\boldsymbol\theta} \operatorname{arg\,max} \ Q(\boldsymbol\theta|\boldsymbol\theta^{(t)}) \,

Note that in typical models to which EM is applied:

  1. The observed data points \mathbf{X} may be discrete (taking one of a fixed number of values, or taking values that must be integers) or continuous (taking a continuous range of real numbers, possibly infinite). There may in fact be a vector of observations associated with each data point.
  2. The missing values (aka latent variables) Y are discrete, drawn from a fixed number of values, and there is one latent variable per observed data point.
  3. The parameters are continuous, and are of two kinds: Parameters that are associated with all data points, and parameters associated with a particular value of a latent variable (i.e. associated with all data points whose corresponding latent variable has a particular value).

However, it is possible to apply EM to other sorts of models.

The motivation is as follows. If we know the value of the parameters \boldsymbol\theta, we can usually find the value of the latent variables Y by maximizing the log-likelihood over all possible values of Y, either simply by iterating over Y or through an algorithm such as the Viterbi algorithm for hidden Markov models. Conversely, if we know the value of the latent variables Y, we can find an estimate of the parameters \boldsymbol\theta fairly easily, typically by simply grouping the observed data points according to the value of the associated latent variable and averaging the values, or some function of the values, of the points in each group. This suggests an iterative algorithm, in the case where both \boldsymbol\theta and Y are unknown:

  1. First, initialize the parameters \boldsymbol\theta to some random values.
  2. Compute the best value for Y given these parameter values.
  3. Then, use the just-computed values of Y to compute a better estimate for the parameters \boldsymbol\theta. Parameters associated with a particular value of Y will use only those data points whose associated latent variable has that value.
  4. Finally, iterate until convergence.

The algorithm as just described will in fact work, and is commonly called hard EM. The K-means algorithm is an example of this class of algorithms.

However, we can do somewhat better by, rather than making a hard choice for Y given the current parameter values and averaging only over the set of data points associated with a particular value of Y, instead determining the probability of each possible value of Y for each data point, and then using the probabilities associated with a particular value of Y to compute a weighted average over the entire set of data points. The resulting algorithm is commonly called soft EM, and is the type of algorithm normally associated with EM. The counts used to compute these weighted averages are called soft counts (as opposed to the hard counts used in a hard-EM-type algorithm such as K-means). The probabilities computed for Y areposterior probabilities and are what is computed in the E-step. The soft counts used to compute new parameter values are what is computed in the M-step.

總結:

EM is frequently used for data clustering in machine learning and computer vision.

EM會收斂到局部極致,但不能保證收斂到全局最優。

EM對初值比較敏感,通常需要一個好的,快速的初始化過程。

 

這是我的Machine Learning課程,先總結到這里, 下面的工作是做一個GM_EM的總結,多維高斯密度估計!

posted on 2010-10-20 14:44 Sosi 閱讀(2524) 評論(0)  編輯 收藏 引用 所屬分類: Courses

統計系統
青青草原综合久久大伊人导航_色综合久久天天综合_日日噜噜夜夜狠狠久久丁香五月_热久久这里只有精品
  • <ins id="pjuwb"></ins>
    <blockquote id="pjuwb"><pre id="pjuwb"></pre></blockquote>
    <noscript id="pjuwb"></noscript>
          <sup id="pjuwb"><pre id="pjuwb"></pre></sup>
            <dd id="pjuwb"></dd>
            <abbr id="pjuwb"></abbr>
            一区二区三区精品国产| 一区二区三区日韩欧美| 久久精品国产一区二区三 | 狠狠色综合播放一区二区| 久久成人国产精品| 久久九九有精品国产23| 亚洲国产一区二区精品专区| 亚洲成人资源网| 欧美日韩天堂| 久久久www成人免费毛片麻豆| 欧美在线视频一区二区三区| 在线成人亚洲| 99亚洲一区二区| 狠狠干狠狠久久| 亚洲国产成人av好男人在线观看| 欧美日韩大片| 久久久亚洲国产天美传媒修理工 | 亚洲一区二区综合| 一色屋精品视频免费看| 亚洲国产精品第一区二区| 欧美日韩综合视频网址| 久久视频一区| 欧美性感一类影片在线播放 | 亚洲视频播放| 久久男人资源视频| 亚洲自拍偷拍视频| 免费观看成人www动漫视频| 亚洲一区国产视频| 免费日韩av| 久久久视频精品| 国产精品久久国产愉拍| 欧美国产欧美综合| 国产日韩精品入口| 亚洲美女av电影| 亚洲国产精品成人一区二区| 亚洲一区二区伦理| aa国产精品| 免费91麻豆精品国产自产在线观看| 亚洲一区二区三区精品动漫| 免费不卡中文字幕视频| 欧美在线黄色| 国产精品久久一级| 日韩视频在线播放| 日韩视频免费看| 六月婷婷一区| 久久午夜视频| 国产亚洲网站| 欧美一区二区三区免费大片| 亚洲永久网站| 欧美手机在线| 99精品视频网| 一区二区三区久久久| 美女日韩在线中文字幕| 乱码第一页成人| 黄色欧美日韩| 久久视频在线看| 久久亚洲视频| 一区精品在线播放| 久久久久五月天| 麻豆精品在线播放| 1769国产精品| 六月天综合网| 91久久综合亚洲鲁鲁五月天| 亚洲国产另类久久精品| 久久美女艺术照精彩视频福利播放| 久久午夜av| 亚洲第一页中文字幕| 免费欧美视频| 亚洲另类在线一区| 亚洲一区二区三区四区中文| 欧美亚韩一区| 先锋影音网一区二区| 久久精品国产99| 在线免费观看欧美| 欧美激情视频网站| 亚洲少妇自拍| 久久久国产精品一区| 在线视频观看日韩| 欧美国产日本| 亚洲色图自拍| 麻豆久久久9性大片| 亚洲精品久久久久久久久久久久| 欧美黄色aa电影| 亚洲伊人一本大道中文字幕| 久久不见久久见免费视频1| 伊人久久大香线蕉av超碰演员| 久久婷婷成人综合色| 亚洲精品美女在线观看播放| 亚洲男人的天堂在线观看| 国产日韩欧美自拍| 欧美成人精品在线| 亚洲天天影视| 欧美成人一品| 亚洲一区二区精品在线| 黄色影院成人| 欧美午夜三级| 久久在线免费观看| 一本一本久久a久久精品牛牛影视| 欧美一区二区日韩一区二区| 亚洲电影天堂av| 国产精品theporn| 久久青青草综合| 亚洲一区二区高清| 亚洲国产精品免费| 久久久久9999亚洲精品| a91a精品视频在线观看| 激情国产一区| 国产精品成人aaaaa网站| 久久久中精品2020中文| 正在播放亚洲一区| 亚洲国产免费| 久久亚洲美女| 欧美一区二区视频97| 日韩午夜在线电影| 激情欧美日韩一区| 国产精品亚洲产品| 欧美日韩免费一区| 免费h精品视频在线播放| 欧美一区二区精品在线| 宅男噜噜噜66国产日韩在线观看| 欧美激情1区2区| 久久亚洲综合色| 久久都是精品| 欧美亚洲三级| 亚洲一区二区成人| aⅴ色国产欧美| 亚洲美女视频在线观看| 亚洲第一网站免费视频| 激情国产一区| 韩国av一区二区三区| 国产亚洲精品自拍| 国产农村妇女毛片精品久久麻豆 | 黄色日韩精品| 经典三级久久| 国产综合色产在线精品| 国产欧美精品日韩| 国产三区精品| 国产女精品视频网站免费| 国产欧美精品va在线观看| 国产精品美女xx| 国产欧美va欧美不卡在线| 国产精品伦子伦免费视频| 国产精品久久国产精品99gif| 欧美日韩在线播放一区二区| 欧美日韩一区在线视频| 欧美少妇一区二区| 国产精品久久婷婷六月丁香| 国产精品久久久久久久app| 国产精品国产三级国产| 国产精品美女久久久免费| 国产精品五区| 狠狠色狠狠色综合系列| 在线观看中文字幕亚洲| 91久久精品一区二区三区| 亚洲茄子视频| 亚洲淫性视频| 久久精品水蜜桃av综合天堂| 久久一二三区| 最新日韩中文字幕| 一区二区三区欧美亚洲| 午夜免费日韩视频| 老司机一区二区| 欧美日本成人| 国产情侣久久| 亚洲国产欧美日韩精品| 亚洲视频久久| 久久躁狠狠躁夜夜爽| 亚洲国产精品视频一区| 亚洲视频免费在线观看| 久久se精品一区二区| 蜜乳av另类精品一区二区| 欧美日韩四区| 激情综合久久| 亚洲一区二区三区高清 | 亚洲国产三级网| 亚洲一区二区在线免费观看视频| 久久久久久97三级| 亚洲国产另类久久精品| 亚洲欧美福利一区二区| 另类酷文…触手系列精品集v1小说| 欧美日韩一区二区三区在线看| 国产精品亚洲一区二区三区在线| 在线日韩欧美| 欧美一区二区三区四区在线观看| 欧美成人免费在线观看| 亚洲视频1区2区| 美女精品国产| 国产一区欧美| 亚洲影视中文字幕| 欧美高清在线视频观看不卡| 亚洲一区二区三区乱码aⅴ| 免费在线日韩av| 国内精品久久久久影院色| 亚洲免费视频一区二区| 欧美韩国在线| 久久精品国产第一区二区三区| 国产精品久久久久99| 亚洲老板91色精品久久| 免费亚洲网站| 欧美影院久久久|