im2recipe.csail.mit.eduRecipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images - MIT
im2recipe.csail.mit.edu Profile
im2recipe.csail.mit.edu
Maindomain:mit.edu
Title:Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images - MIT
Description:Learning Cross-modal Embeddings for Cooking Recipes and Food Images -- Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images Amaia Salvador ♰ Nicholas Hynes
Discover im2recipe.csail.mit.edu website stats, rating, details and status online.Use our online tools to find owner and admin contact info. Find out where is server located.Read and write reviews or vote to improve it ranking. Check alliedvsaxis duplicates with related css, domain relations, most used words, social networks references. Go to regular site
im2recipe.csail.mit.edu Information
Website / Domain: |
im2recipe.csail.mit.edu |
HomePage size: | 16.973 KB |
Page Load Time: | 0.529063 Seconds |
Website IP Address: |
128.30.2.143 |
Isp Server: |
Massachusetts Institute of Technology |
im2recipe.csail.mit.edu Ip Information
Ip Country: |
United States |
City Name: |
Cambridge |
Latitude: |
42.365077972412 |
Longitude: |
-71.104522705078 |
im2recipe.csail.mit.edu Keywords accounting
im2recipe.csail.mit.edu Httpheader
Date: Mon, 10 Feb 2020 07:06:33 GMT |
Server: Apache/2.4.7 (Ubuntu) |
Last-Modified: Sat, 06 Jul 2019 19:25:27 GMT |
ETag: "3ce3-58d082c5b77c0" |
Accept-Ranges: bytes |
Content-Length: 15587 |
Keep-Alive: timeout=5, max=100 |
Connection: Keep-Alive |
Content-Type: text/html |
im2recipe.csail.mit.edu Meta Info
content="text/html; charset=utf-8" http-equiv="Content-Type"/ |
content="NksNPfO4SApMtvU2rGHxr4DPan2Uy6Pz-rP9cA0k1mg" name="google-site-verification"/ |
128.30.2.143 Domains
im2recipe.csail.mit.edu Similar Website
Domain |
WebSite Title |
im2recipe.csail.mit.edu | Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images - MIT |
edg.epa.gov | EPA Environmental Dataset Gateway |
orchardlearningsystems.com | Orchard Learning Systems | Online Learning Courses | 800-764-5845 |
ko.englishcentral.com | 온라인 영어 학습 | 영어 비디오 - Interactive Video Learning | EnglishCentral - Online English Learning |
torchlms.com | Learning Management System - Learning Platform | Absorb LMS Software |
wrodell.scootpad.com | Mosaic Adaptive Academic Learning: Delivering personalized, mastery-based learning to every student, |
lrm.fideliseducation.com | Learning Relationship Management [System], LRM, Personalized Learning Plans, Mentoring Software, Coa |
fideliseducation.com | Learning Relationship Management [System], LRM, Personalized Learning Plans, Mentoring Software, Coa |
members.personallifemedia.com | Personal Life Media Learning Center | Home Study Courses and Products for Lifelong Learning |
learning.linkedin.com | Online Learning & Training Platform for Organizations | LinkedIn Learning |
elt.nysut.org | ELT - NYSUT Education and Learning Trust - Professional Learning |
novoed.com | NovoEd: Online Learning Platform | Collaborative Learning for the Enterprise |
learninghaven.com | Online Learning Haven, homeschooling and distance learning resource. |
customsolutions.jblearning.com | Jones & Bartlett Learning Custom Learning Solutions: Your Course | Your Choice |
learning.aeak12online.org | AEA Learning Online: Student Personalized Learning System |
im2recipe.csail.mit.edu Traffic Sources Chart
im2recipe.csail.mit.edu Alexa Rank History Chart
im2recipe.csail.mit.edu Html To Plain Text
Learning Cross-modal Embeddings for Cooking Recipes and Food Images -- Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images Amaia Salvador ♰ Nicholas Hynes ✦ Aritro Biswas ✦ Yusuf Aytar ✦ Javier Marin ✦ Ferda Ofli ✥ Ingmar Weber ✥ Antonio Torralba ✦ ♰ Universitat Politecnica de Catalunya ✦ Massachusetts Institute of Technology ✥ Qatar Computing Research Institute CVPR 2017 -- * contributed equally -- -- -- -- Download Paper -- -- -- Abstract In this paper, we introduce Recipe1M+, a new large-scale, structured corpus of over one million cooking recipes and 13million food images. As the largest publicly available collection of recipe data, Recipe1M+ affords the ability to train high-capacity modelson aligned, multimodal data. Using these data, we train a neural network to learn a joint embedding of recipes and images that yieldsimpressive results on an image-recipe retrieval task. Moreover, we demonstrate that regularization via the addition of a high-levelclassification objective both improves retrieval performance to rival that of humans and enables semantic vector arithmetic. We postulatethat these embeddings will provide a basis for further exploration of the Recipe1M+ dataset and food and cooking in general. Code, dataand models are publicly available. Check out our most recent journal paper for full details and more analysis! Our CVPR paper can be downloaded from here . Recipe1M+ dataset We present the large-scale Recipe1M+ dataset which contains one million structured cooking recipes with 13M associated images. Follow this link to download the dataset. Below are the dataset statistics: Joint embedding We train a joint embedding composed of an encoder for each modality (ingredients, instructions and images). Results im2recipe retrieval We evaluate all the recipe representations for im2recipe retrieval. Given a food image, the task is to retrieve its recipe from a collection of test recipes. Comparison with humans In order to better assess the quality of our embeddings we also evaluate the performance of humans on the im2recipe task. Check out our most recent journal paper for full details and more analysis. Our CVPR paper can be downloaded from here . -- Examples Demo -- [Coming soon !] -- Check out our online demo , in which you can upload your food images to retrieve a recipe from our dataset. -- Demo [Coming soon !] Check out our online demo , in which you can upload your food images to retrieve a recipe from our dataset. Embedding Analysis We explore whether any semantic concepts emerge in the neuron activations and whether the embedding space has certain arithmetic properties. Visualizing embedding units We show the localized unit activations in both image and recipe embeddings. We find that certain units show localized semantic alignment between the embeddings of the two modalities. Arithmetics We demonstrate the capabilities of our learned embeddings with simple arithmetic operations. In the context of food recipes, one would expect that: v(chicken\_pizza) - v(pizza) + v(lasagna) = v(chicken\_lasagna) where v represents the map into the embedding space. We investigate whether our learned embeddings have such properties by applying the previous equation template to the averaged vectors of recipes that contain the queried words in their title. The figures below show some results with same and cross-modality embedding arithmetics. Image Embeddings Recipe Embeddings Fractional arithmetics Another type of arithmetic we examine is fractional arithmetic, in which our model interpolates across the vector representations of two concepts in the embedding space. Specifically, we examine the results for: where varies from 0 to 1. Image Embeddings Recipe Embeddings Embedding visualization In order to visualize the learned embedding we make use of t-SNE and the 50k recipes with nutritional information we have within Recipe1M+. Semantic categories In the figure bellow we show those recipes that belong to the top 12 semantic categories used in our semantic regularization. Healthiness within the embedding In the next figure we can see the previous embedding visualization but this time showing the same recipes on different colors depending on how healthy they are in terms of sugar, fat, saturates and salt. Code & Trained Models Code on GitHub Pretrained Model -- Pretrained Model Code on GitHub (Pytorch) Pretrained Model (Pytorch) Citation @article{marin2019learning, title = {Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images}, author = {Marin, Javier and Biswas, Aritro and Ofli, Ferda and Hynes, Nicholas and Salvador, Amaia and Aytar, Yusuf and Weber, Ingmar and Torralba, Antonio}, journal = {{IEEE} Trans. Pattern Anal. Mach. Intell.}, year = {2019} } @inproceedings{salvador2017learning, title={Learning Cross-modal Embeddings for Cooking Recipes and Food Images}, author={Salvador, Amaia and Hynes, Nicholas and Aytar, Yusuf and Marin, Javier and Ofli, Ferda and Weber, Ingmar and Torralba, Antonio}, booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, year={2017} } Acknowledgements This work has been supported by CSAIL-QCRI collaboration projects and the framework of projects TEC2013-43935-R and TEC2016-75976-R, financed by the Spanish Ministerio de Economia y Competitividad and the European Regional Development Fund (ERDF)....