Summary Fashion Attributes

Clothing Recommender System

• Part I : Object Detection
• Part II : Attribute Tagging
• Part 3 : Recommendation based on Frequency

Fashion Meets Computer Vision: A Survey

Paper #1 - Progressive Fashion Attribute Extraction

• Attributes (neck design detailing, sleeves detailing, etc) 

Paper #2 - Attr2Style: A Transfer Learning Approach for Inferring Fashion Styles via Apparel Attributes

• Low-level attributes of an apparel (for example, neck type, dress length, collar type, print etc)

• Transfer learning based approach to address the issue of style-based image captioning for a target dataset

Paper #3 - The iMaterialist Fashion Attribute Dataset

Paper #4 - A Deep-Learning-Based Fashion Attributes Detection Model

Paper #5 - FashionSearchNet-v2: Learning Attribute Representations with Localization for Image Retrieval with Attribute Manipulation

Myntra Customization

Occasion based Recommendation system in E-commerce like Amazon, Etsy

Visual Attributes for Fashion Analytics

We use low-level visual features to predict intermediate clothing attributes such as color, pattern, material, or collar type Occasion-oriented clothing recommendation

Attribute Types

Color/ Attributes

More Reads

Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning

Keep Exploring!!!

Fashion - Papers - Vision and Fashion

Paper #1 - Session-based Complementary Fashion Recommendations

Key Notes

• Sampling strategy adopted to build the training set
• A/B test carried out in a fashion e-commerce platform with 28 million active customers.
• ”Perfect pairings: You might also like”. ‡ese type of recommendations allow customers to continue the fashion discovery journey by €finding items that can be worn together
• Worn-together concept: two items are complementary if they can be worn together.
• Collaborative €filltering approach based on cosine similarity
• Click through rate (CTR) measure
• Session-based Recommendations
• Combine both stylistic and functional facets of complementary items across categories
• Session-based approaches that follow a Recurrent Neural Networks architecture
• STAMP, a novel Short-Term AŠention Priority Model for Session-based Recommendation
• Customers’ response to the baseline recommender, and training the model to maximize the accuracy of the next-click prediction
• Cross-sell sequences selected from the users’ interaction histories
• Items purchased by users are intuitively more representative for their long-term tastes and preferences.
• Categorical features coming from the metadata of an item such as the category
• The training set consists of 5073130 cross-sell examples from
• 1195512 users with an average length of 13.08

Key observations (Inspirations)

• Get recent browse / search history
• Get recent cross-category search/purchases
• Look at patterns in combinations for search / past purchase / cross category views
• Club of all of them for Work together / perfect pairing

Paper #2 - +GAN: Complementary Fashion Item Recommendation

Key Notes

• Generative adversarial model to draw realistic samples from paired fashion clothing distribution
• Dataset from web crawled data
• bidirectional LSTM model to sequentially predict the next item
• A conditional GAN can be trained to fill up this missing piece by generating possible clothing choices which can be subsequently
• Discrete cosine transform (DCT) 
• K-means based clustering on the intensity field of the images

Paper #3 - Fashion Recommendation Based on Style and Social Events

Key Notes

• Semantic layer is based on the style of the proposed dressing
• Mood and the emotion concealed behind color combination patterns
• Patterns behind color combinations have been extensively studied
• Fashion4Events dataset comprising approximately 400k garment images with social event labels
• DeepFashion2 is a dataset that proposed a unified benchmark for clothes detection, segmentation, retrieval, and landmark prediction
• Color classifier and the event classifier in our recommendation system
• Visual-semantic embedding and training a bidirectional LSTM (Bi-LSTM) model to sequentially predict the next item

Paper #4 - A method for extracting emotion using colors comprise the painting image

Key Notes

Colors to Emotions

• VisualFashionAttributePrediction
• Fashion Category and Attribute Prediction!
• Disentangled Representations for Interactive Fashion Retrieval
• Shopping100k: contact the author of the dataset to get access to the images.
• DeepFashion: download images and labels for the category and attribute prediction benchmark from the dataset website.
• Clothing_TAN
• Fashion Attribute Detection
• Large-scale Fashion (DeepFashion) Database
• DeepFashion2 Dataset
• DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations
• Clothes Type Recognition with DeepFashion Dataset and Fast AI

Detectron2 Beginner's Tutorial

Detectron2 

Monk Object Detection - A low code wrapper over state-of-the-art deep learning algorithms

Keep Exploring!!!

Virtual Try on - Fashion - Paper - Reading Notes

• A Curated List of Awesome Virtual Try-on (VTON) Research
• Dressing in Order (DiOr)
• virtual_try_on_use_deep_learning
• DeepFashionSwap
• Self-Correction for Human Parsing
• Instance-level Human Parsing via Part Grouping Network
• Deep Virtual Try-on with Clothes Transform
• Dense Pose
• Image-Based Virtual Fitting System with Garment Image Reshaping

Key Techniques

• Landmarks
• Segmentation
• Masking
• Target addition

Approach #1

Ref1

Approach #2

Ref 

Approach 3

Ref

Papers

• SwapNet
• Down to the Last Detail: Virtual Try-on with Detail Carving
• Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image Content
• Toward Characteristic-Preserving Image-based Virtual Try-On Network

Keep Exploring!!!

Interesting poster sessions

Vision Fashion Papers

Paper#1 - POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion

Key Notes

• Personalized Out€fit Generation (POG) model = user preferences
• regarding individual items +  outfi€ts with Transformer architecture
• Personalization represents how the recommendations meet users’ personal fashion tastes

• Key features - brand, category, style, pattern
• Fashion Outfit Model (FOM) by learning the compatibilities between each item and all the other items within the outfit
• Personalized Outfit Generation (POG) model, which can generate compatible and personalized outfits based on users’ recent behaviors

• Combination of NLP, Vision, Graph Embedding

• This could be a combination of user-user, item-item and bought sequences of complete pairs

Paper #2 - MMFashion: An Open-Source Toolbox for Visual Fashion Analysis

Key Notes

• Fashion Attribute Prediction, Fashion Recognition and Retrieval, Fashion Landmark Detection, Fashion Parsing and Segmentation and Fashion Compatibility and Recommendation.
• Dataset - DeepFashion, Polyvore
• Clothes Retrieval
• Landmark Detection
• Cloth Detection and Segmentation
• Fashion Compatibility and Recommendation

Paper#3 - c+GAN: Complementary Fashion Item Recommendation

Key Notes

• Bidirectional LSTM model to sequentially predict the next item conditioned on previous ones
• Clustering the intensity field of the images, with K-means clustering results in these dominant clusters
• Combination of Text + Vision Similarity + GAN would be good

More Reads

CRAFT: Complementary Recommendation by Adversarial Feature Transform

Keep Reading. This is just very basic skimming!!!!

Fashion Paper Reads - Part II

Paper #1 - Detailed Garment Recovery from a Single-View Image

Key Notes

• Global shape and geometry of the clothing
• Extract occluded wrinkles and folds
• Parameter estimation, semantic parsing, shape recovery, and physics-based cloth simulation

The current implementation of our approach depends on two

• databases: a database of commonly available garment templates
• database of human-body models.

Implementation Notes

• 14 joint positions on the image and provides a rough sketch outlining the human body silhouette
• Semantic parse of the garments in the image to identify and localize depicted clothing items

Human body - We follow the PCA encoding of the human body shape presented in [Hasler et al. 2009]. The semantic parameters include gender, height, weight, muscle percentage, breast girth, waist girth, hip girth, thigh girth, calf girth, shoulder height, and leg length

Garment Parsing

• we extract the clothing regions Ωb,h,g by performing a two-stage image segmentation guided by user sketch
• Initial garment registration results. We fit garments to human bodies with different body shapes and poses.

Implementation - We have implemented our algorithm in C++ and demonstrated the effectiveness of our approach throughout the paper

Paper #2 - M2E-Try On Net: Fashion from Model to Everyone

• Pose alignment network (PAN) - pose alignment network (PAN) to align the model and clothes pose to the target pose
• Texture refinement network (TRN) - enrich the textures and logo patterns to the desired clothes
• Fitting network (FTN) - merge the transferred garments to the target person images.
• Unsupervised learning and self supervised learning to accomplish this task.
• Generative adversarial network (GAN) [9] has been used for image-based generation
• GAN has been used for person image generation [18] to generate the human image from pose representation
• For fashion image generation, a more intuitive way is to generate images from a person image and the desired clothes image

Dataset - Deep Fashion [17] Women Tops dataset, MVC [16] Women Tops dataset and MVC [16]

PAN 

• PAN as a conditional generative module
• To train PAN, ideally we need to have a training triplet with paired images: model image M, person image P, and pose aligned model image
• self-supervised training method that uses images of the same person in two different poses to supervise Pose Alignment Network (PAN)

Texture Refinement Network (TRN)

• Combine the information from network generated images and texture preserved images produced by geometric transformation
• Loss - Reconstruction loss, perceptual loss and style loss are used only for paired training

Paper #3 - VITON-GAN: Virtual Try-on Image Generator

Trained with Adversarial Loss

Code - Link

Paper #4 - GarmentGAN: Photo-realistic Adversarial Fashion Transfer

This method divides the image generation task into two sub-tasks: segmentation map synthesis and transference of the clothing characteristics onto the previously generated map

The system comprises two separate GANs: a shape transfer network and an appearance transfer network

More Reads

• GarmentGAN: Photo-realistic Adversarial Fashion Transfer
• Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data
• Algorithmic clothing: hybrid recommendation, from street-style-to-shop
• Aesthetic-based Clothing Recommendation

Keep Thinking!!!

Weekend Reading - Fashion Research Paper Reads # Part 1

Paper #1 - Outfit Recommender System

Perceptions

• Christian funeral, wearing black conservative clothes is customary
• Hindu funeral, wearing white conservative clothes is the norm

A recommender system is used to suggest products to customers by using information about the customer. Feature Descriptors - Histogram of Oriented Gradients (HOG), Speeded Up Robust Feature (SURF), Scale Invariant Feature Transform (SIFT)

• Event Analysis using Detected Objects
• Fifty-three categories of clothes

1. Multilabel classification and cloth detection
2. Approach Events - Clothes Mapping
3. Event to Outfit Mapping

Crop the images to reduce the background information

Paper #2 - Artificial Intelligence and the Fashion Industry

• Datafication of fashion - data generation through the digitization of content,and monitoring of activities, including real world activities and phenomena,through sensors
• Image Search. Personalized shopping is also achieved using AI applications based on computer vision and augmented and virtual reality.
• Reverse image search instead is the process by which an image is used to find another image.
• Visual search, a subset of reverse image search, refers to the possibility of finding items within an image and searching for those.

Paper #3 - Recommendation based on multiproduct utility maximization

Multi-product utility maximization (MPUM) - integrates the economic theory of consumer choice theory with a personalized recommendation

Two products could be substitutes - buy A instead of B or complements - buy A together B. Identifying and making use of such relationships are useful for recommendation systems.

We assume user make choices to maximize multi-product utility and use the multinomial consumer choice model for that, then the multi-product utility model can be learned to maximize the likelihood of observed user data

Collaborative filtering is based on the assumption that users with similar tastes for previous items will have similar preferences for new items

Content-based filtering is based on the assumption that the features (meta data, words in the description, price, tags, visual features, etc.) 

Hybrid recommendation algorithms combine collaborative filtering with content-based filtering

Code - Link1, Link2

Keep Learning!!!

Weekend Reads - Fashion Virtual Dress Creation - Papers

Paper #1 - M2E-Try On Net: Fashion from Model to Everyone
Key Notes

• Pose alignment network (PAN)
• Texture refinement network (TRN)
• Fitting network (FTN)

Pose alignment network - (PAN) to align the model and clothes pose to the target pose. Each dense pose prediction has a partition of 24 parts

Texture refinement network (TRN) to enrich the textures and logo patterns to the desired clothes
Texture details, region of texture, binary mask with the same size. Merged images while still preserving the textual details on the garments

Fitting network (FTN) to merge the transferred garments to the target person images. Generative network to generate fashion images from textual inputs

Fitting Network is an encoder-decoder network, including three convolution layers as the encoder, six residual blocks for feature learning, followed by two deconvolution layers and one convolution layer as the decoder

Code - https://github.com/shionhonda/viton-gan

Paper #2 - VITON-GAN: Virtual Try-on Image Generator Trained with Adversarial Loss
Key Notes

• U-net generator and thin plate spline (TPS)
• Human parser
• Pose estimator

GANs are able to generate fine, high-resolution, and realistic images because adversarial loss can incorporate perceptual features that are difficult to define mathematically

Try-on module (TOM)

• Trained adversarially against the discriminator that uses the TOM result image
• Person representation as inputs and judges whether the result is real or fake VITON-GAN generated hands and arms

Paper #3 - The Conditional Analogy GAN: Swapping Fashion Articles on People Images

Key Notes

• GAN - a model G that learns a data distribution from example data, and a discriminator D that attempts to distinguish generated from training data
• These models learn loss functions that adapt to the data, and this makes them perfectly suitable for image-to-image translation tasks

• (cGAN) learns to generate images as function of conditioning information from a dataset, instead of random noise from a prior, as in standard GAN

Training of the CAGAN model involves learning a generator G to generate plausible images which fool a discriminator D. The discriminator D needs to answer two questions:

• does an image x look reasonable, i.e. indistinguishable from the training distribution of human images {xi}?
• does the article y look well-painted on the human model image x

Neural Re-Rendering of Humans from a Single Image

More Reads

GENERATIVE ADVERSARIAL NETWORK-BASED VIRTUAL TRY-ON WITH CLOTHING REGION

Keep Thinking!!!