tag:blogger.com,1999:blog-590043897961646114.post6482624831423838746..comments2024-02-06T02:06:06.364-08:00Comments on Engaging Market Research: Identifying Pathways in the Consumer Decision Journey: Nonnegative Matrix FactorizationUnknownnoreply@blogger.comBlogger13125tag:blogger.com,1999:blog-590043897961646114.post-85243090585625190082015-05-05T17:13:25.086-07:002015-05-05T17:13:25.086-07:00Cool, glad I have it down finally. One thing that ...Cool, glad I have it down finally. One thing that bugged me about reading Skillicorn (and I asked him about it but will confess I did not understand the answer) was I expected that if I ran NMF on a matrix and then ran it on the transpose of that same matrix, I would find the matrices switched and transposed (as I thought he showed on page 174) but in fact that is not the case and the matrices are completely different. Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-60265959003188213292015-05-05T17:07:16.525-07:002015-05-05T17:07:16.525-07:00Well said! I have had problems trying to explain t...Well said! I have had problems trying to explain this point, and your comment will help future readers. Thanks.Joel Cadwellhttps://www.blogger.com/profile/14946447393733294251noreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-55388535160493132022015-05-05T16:46:55.380-07:002015-05-05T16:46:55.380-07:00I have that book actually! How does this sound:
W...I have that book actually! How does this sound:<br /><br />When input is [person x attributes] then W is [person x latent dimensions] and H is [latent dimensions x attributes]. Thus, to get the weightings of each attribute in each latent dimension, look at the columns of H (each attribute is a column and the rows are the latent dimension strength). This is what I think you have done in your post(?) <br /><br />When input is [terms x documents] then W is [terms x latent dimensions (topics)] and H is [latent dimensions x documents]. Thus the columns of W describe the strength of each term for a given latent dimension (each column is a latent dimension). The columns of H show the strength or mix of each latent dimension for a given document (each is a column).Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-72517190561333761222015-05-05T15:45:09.341-07:002015-05-05T15:45:09.341-07:00You can find a discussion in Chapter 8.1 from a bo...You can find a discussion in Chapter 8.1 from a book by David Skillicorn called Understanding Complex Datasets (2007). You may not see that section in Google Books, but there is a book pdf that you can find if you search for it.<br /><br />Try to remember: W for the rows and H for the columns. The latent features are the columns of W and the rows of H. When the data matrix is person x measures of the person, the transpose of H yields something that looks like a matrix of factor loadings with simple structure. W looks like a membership matrix from finite mixture models. Of course, you may need to rescale the values of W and H since W*H reproduces the raw data matrix, which can be counts or any arbitrary non-negative intensity scale.Joel Cadwellhttps://www.blogger.com/profile/14946447393733294251noreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-57485179070240609862015-05-05T11:38:19.431-07:002015-05-05T11:38:19.431-07:00Hey Joel- Great stuff! I am very confused by what ...Hey Joel- Great stuff! I am very confused by what vectors (rows or columns of which matrix (W or H) ) to use for the latent factors, depending on if the input matrix is (for example) documents x terms or terms x documents. I saw your other post but is there any resources you have found that describes this?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-40578713428398599772014-07-16T06:50:52.341-07:002014-07-16T06:50:52.341-07:00Yes, NMF can be used as a pre-processing step for ...Yes, NMF can be used as a pre-processing step for future clustering or regression analysis. And yes, one can impose orthogonality constraints on the factors. You can find lots of examples using the basis matrix in the same way as principal component or factor scores. However, orthogonal NMF has been introduced primarily as a means for dealing with the non-uniqueness of the NMF solution and not because the columns of the basis matrix are collinearity.Joel Cadwellhttps://www.blogger.com/profile/14946447393733294251noreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-51480955955098614002014-07-16T02:05:53.817-07:002014-07-16T02:05:53.817-07:00To build on your example, say I want to look at fa...To build on your example, say I want to look at factors associated with purchase likelihood, for example, using a logistic regression. I could have used the individual episodes (original columns of your data matrix) as dependent variables, but this isn't a particularly parsimonious representation and so much detail may obscure the main story (e.g online shoppers have higher purchase likelihood than offline shoppers):<br /><br />1) Can I use the columns of the basis matrix as independent variables in a regression?<br />2) When running NMF, can you choose to enforce orthogonality to avoid multicollinearity due to correlations between basis columns?<br /><br />Thanks in advance!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-30847283388554155072014-07-15T10:52:13.461-07:002014-07-15T10:52:13.461-07:00Thanks this is super helpful!Thanks this is super helpful!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-7068859846531442672014-07-14T05:44:21.458-07:002014-07-14T05:44:21.458-07:00Let me answer your last two questions first. NMF h...Let me answer your last two questions first. NMF has become an established technique, as you can see from all the references on the Wikipedia link. The onlilne textbook "Elements of Statistical Learning Theory" provides a good introduction. Li and Ding have a recent book chapter called "Non-negative Matrix Factorization for Clustering: A Survey" at http://users.cis.fiu.edu/~taoli/pub/home-new.html. Nicolas Gillis "The Why and How of Nonnegative Matrix Factorization" at http://arxiv.org/pdf/1401.5226v2.pdf might help. Ankur Moitra has a YouTube video called "New Algorithms for Nonnegative Matrix Factorization and Beyond" with slides at his website.<br /><br />Now, deciding how many latent components is best depends on your definition of best. It is decided in the same manner as the number of clusters or number of factors question in cluster and factor analysis. Are additional latent components meaningful and interpretable or are they noise? Let's run several different ranks and take a look. Section 2.6 of the NMF introductory vignette discusses these issues under the heading "Estimating the Factorization Rank" where rank is the number of components.<br /><br />I hope this helps. Good luck.<br />Joel Cadwellhttps://www.blogger.com/profile/14946447393733294251noreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-24311516988775711242014-07-14T02:15:49.641-07:002014-07-14T02:15:49.641-07:00Hi, would appreciate any comments on:
1) How to de...Hi, would appreciate any comments on:<br />1) How to decide how many latent components 'best' describe the data?<br />2) Any situations in which NMF may be inappropriate? I'm a little leery of applying techniques I don't have a deep mathematical understanding of to new situations?<br />3) The (intuitive) difference between applying NMF to a data matrix and, say, PCA or classical MDS?<br /><br />Thanks!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-82480476050640504592014-06-26T03:51:00.141-07:002014-06-26T03:51:00.141-07:00Hi, Great post as always. You're one of the fe...Hi, Great post as always. You're one of the few people who describe marketing analysis so well using R. Please keep this up! Cheers!Unknownhttps://www.blogger.com/profile/00675716111484535267noreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-52228134654976482462014-06-16T06:48:31.545-07:002014-06-16T06:48:31.545-07:00Correction made, thanks.Correction made, thanks.Joel Cadwellhttps://www.blogger.com/profile/14946447393733294251noreply@blogger.comtag:blogger.com,1999:blog-590043897961646114.post-43946506949926290882014-06-15T07:49:38.914-07:002014-06-15T07:49:38.914-07:00Hey Joel - I am an editor who is studying to becom...Hey Joel - I am an editor who is studying to become a data scientist and I noticed a typo in the R program. The fourth to last item in the c() command should be "Voucher," not "Vocher." Very interesting article, by the way. Regards.Roufa Therrienhttps://www.blogger.com/profile/16526170979479333875noreply@blogger.com