Hello Daniel, Shabir here first of all amazing work and I have been following this project from the start . The problem I faced was when I opened the link on my android device , and clicked on browse files I couldn't upload an image by capture , it took me to the screen to select photo from my mobile storage and also I noticed the page didn't refresh after I uploaded. I could see my image and other things as it is with the upload success message . Thanks again!

Every time there's a prediction, there should be an easy to use button asking "is this correct?".

The result of tapping this button will update a database prediction to say whether or not the prediction was right or wrong.

This will help evaluate the model(s) in the wild.

Tried installing pre-commit (https://pre-commit.com), it looks like it works for some files but breaks others when I try to commit them.

Because the tests don't pass they never commit.

For example:

flake8...................................................................Failed
- hook id: flake8
- exit code: 1

food_image_collector.py:15:80: E501 line too long (97 > 79 characters)
food_image_collector.py:49:80: E501 line too long (80 > 79 characters)
food_image_collector.py:54:80: E501 line too long (82 > 79 characters)
food_image_collector.py:59:80: E501 line too long (123 > 79 characters)
food_image_collector.py:66:80: E501 line too long (80 > 79 characters)
food_image_collector.py:67:80: E501 line too long (81 > 79 characters)
food_image_collector.py:123:80: E501 line too long (114 > 79 characters)
food_image_collector.py:129:80: E501 line too long (106 > 79 characters)
food_image_collector.py:134:80: E501 line too long (84 > 79 characters)
food_image_collector.py:136:80: E501 line too long (102 > 79 characters)
food_image_collector.py:137:80: E501 line too long (84 > 79 characters)
food_image_collector.py:144:80: E501 line too long (91 > 79 characters)

mypy.....................................................................Passed

See the initial setup here: https://github.com/mrdbourke/nutrify/issues/1#issuecomment-916722957

Uncaught SyntaxError: "0"-prefixed octal literals are deprecated; use the "0o" prefix instead

This may be a known issue. This is console output when accessing the webpage.

I think this results in the upload not working (unless I'm mistaken).

The problem is in constants.js with egg tart having an id starting with 0.

Hope this helps. Keep up the work.

I heard something about CI/CD.

Github action is good for this.

I have no idea how it works.

So I'm keen to tag along with this.

Edit: The idea is you commit your code and this will automatically update the production code. No downtime, all in one automated step.

Noticed the TODO in evaluate.py script and thought to make your work easier. Made a function to set random seeds for random and torch module. Also made some styling changes in the script to go hand-in-hand with conventions. For example,

# import standard python libraries
# import from 3rd party libraries (numpy, pandas, torch etc.)
# import from source code (code that you have written, like `utils` etc.)

Hope this branch gets merged and helps in the making of this project!

There should be a food/not food model running with the app.

This is so the app can detect whether food is present or not and then proceed with the classification of what food is present.

Was looking up some domains. https://www.nutrify.com/ - this is taken and it's a registered name. So does this mean the whole projects needs a new name? I hope not...

Related PR: #4

I understand that we want simplicity, but it is also a good idea to slowly include coding best practices...

• incorporate pytest for automated testing
• also have pre-commit hooks for linting, (mypy, flake8) and code formating black.
• switch to pipenv for package management instead of a requirements.txt, we can manage production packages (e.g. streamlit) and the above dev pages separately.

Some resources:

• https://docs.pytest.org/en/6.2.x/
• https://shivansivakumaran.com/coding/my-tubestats-app/#Hurdle_6_how_to_test_code
• https://ljvmiranda921.github.io/notebook/2018/06/21/precommits-using-black-and-flake8/
• https://pipenv.pypa.io/en/latest/

Starting to build out functionality for Nutrify's data flywheel.

This PR adds train.py and evaluate.py as major scripts for training a FoodVision model as well as evaluating it.

There's also a new directory called foodvision, this is where all of Nutrify's ML related files will go (for now) when it comes to training a computer vision model for identifying foods.

Next will be to create a script to import the "100 most wrong predictions" into Label Studio to update and improve the labels before retraining another model.

Tried to make the website a little more better !. Added CSS and used Bootstrap to make the website looks more better.

Screen Shots of the new looking Nutrify Website.

I've currently got a workflow for:

• training a model in train.py
• evaluating the trained model in evaluate.py

evaluate.py stores the "X most wrong" predictions in a CSV/Weights & Biases Table/Artifact, so next will be to pull that information into a script such as fix_labels.py which:

• inputs: a CSV file of "X most wrong" predictions (their labels, their images etc)
• outputs: a Label Studio labelling task to fix/update the labels

There could be a few options in the Label Studio interface to make the dataset better:

1. confusing/clear - a label to state whether the image is confusing (e.g. multiple foods, lots going on, poor image) or clear (e.g. a single food with a good picture)
2. whole_food/dish - a label to state whether the image has a single food or multiple foods in it (can use this later to differentiate between dishes and whole foods)
3. prediction/updated class - a label which represents the updated information about the image

I'd like a script to create 3 different models (or more):

1. Big - can be used for labelling/label improvements, this model will be the biggest and best but will be almost impossible to deploy properly, see here: https://huggingface.co/BAAI/EVA (this model requires a GPU with at least 24GB for batches of 16)
2. Medium - can be used for experimenting, reviewing labels, etc
3. Small - smallest and fastest model used for deploying (this model won't be as accurate as the others but will definitely be the fastest/capable of running on smaller devices/in the browser), see here: https://pytorch.org/vision/stable/models/generated/torchvision.models.quantization.shufflenet_v2_x2_0.html#shufflenet-v2-x2-0

Ideas

I could have a Python script called create_models.py which contains 3x functions (one for each model).

Then in the train.py script, I can call the appropriate model for the job.

Could use the "strategy" pattern in Python, see this video: https://youtu.be/WQ8bNdxREHU

Every image uploaded to Nutrify should be saved to storage (as long as the image is detected as being food).

The image should be uploaded -> passed through food/not_food model -> saved to storage if the image if of food, then track:

• Image UUID - use this to track information regarding different images across the database
• Timestamp
• Pred for image class

Going to have to look into page loading times for this one.

I'd like the upload image function to be available as soon as the page loads.

Right now it's about a second or two behind the page itself.

Not a huge deal but experience wise, it could be better.

I'd like to be able to add an FDC ID to some section or a single file and that file gets watched.

Every time a new FDC ID gets added to that file, information from the Food Data Central Database gets pulled in to be used Nutrify style.

For example:

• New FDC ID added to a list
• List gets watched and update found -> triggers function
• Function goes to FDC database and pulls information based on the ID
• Nutrition information from that food ID is placed in the Nutrify database to be displayed on the web

Some classes are missing FDC data and will have to be fixed later on.

Need a way to:

• Know what classes a model has been trained on
• Sync up model classes with food data (the data from the FDC)
• Only publish models that have accompanying food data with them

This will solve the problem of someone taking a photo of something an data not being displayed.

Or...

1. Create a model with X amount of classes
2. Make dummy FDC data for the classes that don't have it yet
3. Display information for which classes have data and which classes don't