Skip to content

Commit c336250

Browse files
xta0Pat Mellon
authored andcommitted
update iOS.md
1 parent b38e4c0 commit c336250

File tree

1 file changed

+38
-19
lines changed

1 file changed

+38
-19
lines changed

_mobile/ios.md

Lines changed: 38 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -10,42 +10,44 @@ published: true
1010

1111
# iOS
1212

13-
To get started with PyTorch on iOS, we recommend exploring the following a HelloWorld example on Github [Hello Word](https://github.com/pytorch/ios-demo-app/tree/master/HelloWorld).
13+
To get started with PyTorch on iOS, we recommend exploring the following [HelloWorld](https://github.com/pytorch/ios-demo-app/tree/master/HelloWorld).
1414

15-
## Quickstart with a HelloWorld example
15+
## Quickstart with a Hello World example
1616

17-
HelloWorld is a simple image classification application that demonstrates how to use PyTorch C++ libraries on iOS. The code is written in Swift and uses an Objective-C class as a bridging header.
17+
HelloWorld is a simple image classification application that demonstrates how to use PyTorch C++ libraries on iOS. The code is written in Swift and uses Objective-C as a bridge.
1818

19-
Before we jump into details, we highly recommend following the Pytorch Github page to set up the Python development environment on your local machine.
19+
Before we jump into details, we highly recommend following the [Pytorch Github page](https://github.com/pytorch/pytorch) to set up the Python development environment on your local machine.
2020

2121
### Model preparation
2222

23-
Let's start with model preparation. If you are familiar with PyTorch, you probably should already know how to train and save your model. In case you don't, we are going to use a pre-trained image classification model(Resnet18), which is packaged in [TorchVision](https://pytorch.org/docs/stable/torchvision/index.html). To install TorchVision, run the command below.
23+
Let's start with model preparation. If you are familiar with PyTorch, you probably should already know how to train and save your model. In case you don't, we are going to use a pre-trained image classification model(Resnet18), which is packaged in [TorchVision](https://pytorch.org/docs/stable/torchvision/index.html). To install it, run the command below.
2424

2525
```shell
2626
pip install torchvision
2727
```
2828

29-
Once we have TorchVision installed successfully, let's navigate to the HelloWorld folder and run a python script to generate our model. The `trace_model.py` contains the code of tracing and saving a [torchscript model](https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html) that can be run on mobile devices. Run the command below to get our model
29+
Once we have TorchVision installed successfully, let's navigate to the HelloWorld folder and run `trace_model.py` to generate our model. The script contains the code of tracing and saving a [torchscript model](https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html) that can be run on mobile devices. Run the command below to get our model
3030

3131
```shell
3232
python trace_model.py
3333
```
3434

3535
If everything works well, we should have our model - `model.pt` generated in the same folder. Now copy the model file to our application folder `HelloWorld/model`.
3636

37+
More details about TorchScript you can find in [tutorials on pytorch.org](https://pytorch.org/docs/stable/jit.html)
38+
3739
### Install PyTorch C++ libraries via Cocoapods
3840

3941
The PyTorch C++ library is available in [Cocoapods](https://cocoapods.org/), to integrate it to our project, we can run
4042

4143
```ruby
4244
pod install
4345
```
44-
Now it's time to open the `HelloWorld.xcworkspace` in XCode, select an iOS simulator and hit the build and run button (cmd + R). If everything works well, we should see a wolf picture on the simulator screen along with the prediction result.
46+
Now it's time to open the `HelloWorld.xcworkspace` in XCode, select an iOS simulator and launch it (cmd + R). If everything works well, we should see a wolf picture on the simulator screen along with the prediction result.
4547

4648
### Code Walkthrough
4749

48-
In this part, we are going to walk through the code step by step. The `ViewController.swift` contains most of the code.
50+
In this part, we are going to walk through the code step by step. All logic happens in `ViewController.swift`.
4951

5052
#### Image loading
5153

@@ -60,7 +62,7 @@ guard var pixelBuffer = resizedImage.normalized() else {
6062
}
6163
```
6264

63-
We first load an image from the bundle and resize it to 224x224, which is the size of the input tensor. Then we call this `normalized()` category method on UIImage to get normalized pixel data from the image. Let's take a look at the code below.
65+
We first load the image from our bundle and resize it to 224x224. Then we call this `normalized()` category method to normalized the pixel buffer. Let's take a closer look at the code below.
6466

6567
```swift
6668
var normalizedBuffer: [Float32] = [Float32](repeating: 0, count: w * h * 3)
@@ -72,11 +74,12 @@ for i in 0 ..< w * h {
7274
normalizedBuffer[w * h * 2 + i] = (Float32(rawBytes[i * 4 + 2]) / 255.0 - 0.406) / 0.225 // B
7375
}
7476
```
75-
The input data of our model is a 3-channel RGB image of shape (3 x H x W), where H and W are expected to be at least 224. The image have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225].
7677

77-
#### Init JIT interpreter
78+
The code might look weird at first glance, but it’ll make sense once we understand our model. The input data of our model is a 3-channel RGB image of shape (3 x H x W), where H and W are expected to be at least 224. The image have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225].
79+
80+
#### TorchScript module
7881

79-
Now that we have preprocessed our input data and we have a pre-trained TorchScript model, the next step is to use the model and the data to run the predication. To do that, we'll first load our model into the application.
82+
Now that we have preprocessed our input data and we have a pre-trained TorchScript model, the next step is to use them to run predication. To do that, we'll first load our model into the application.
8083

8184
```swift
8285
private lazy var module: TorchModule = {
@@ -88,25 +91,43 @@ private lazy var module: TorchModule = {
8891
}
8992
}()
9093
```
91-
The TorchModule Class is an Objective-C wrapper for the C++ class `torch::jit::script::Module`.
94+
Note that the `TorchModule` Class is an Objective-C wrapper of `torch::jit::script::Module`.
9295

9396
```cpp
9497
torch::jit::script::Module module = torch::jit::load(filePath.UTF8String);
9598
```
99+
Since Swift can not talk to C++ directly, we have to either use an Objective-C class as a bride, or create a C wrapper for the C++ library. For the demo purpose, we're going to wrap everything in this Objective-C class, but we are working on bringing the Swift wrapper to PyTorch.
96100

97101
#### Run Inference
98102

99-
Now it's time to run the inference and get the result. We pass in the pixel buffer object as a raw pointer to the `predict` method and get the result from it.
103+
Now it's time to run the inference and get the results.
100104

101105
```swift
102106
guard let outputs = module.predict(image: UnsafeMutableRawPointer(&pixelBuffer)) else {
103107
return
104108
}
105109
```
106-
Again, the `predict` method on the `module` is an Objective-C method. Under the hood, it calls the C++ version of predict which is `forward`
110+
Again, the `predict` method is just an Objective-C wrapper. Under the hood, it calls the C++ `forward` function. Let's take a look at how it's implemented.
111+
112+
```cpp
113+
at::Tensor tensor = torch::from_blob(imageBuffer, {1, 3, 224, 224}, at::kFloat);
114+
torch::autograd::AutoGradMode guard(false);
115+
at::AutoNonVariableTypeMode non_var_type_mode(true);
116+
auto outputTensor = _impl.forward({tensor}).toTensor();
117+
void* tensorBuffer = outputTensor.storage().data();
118+
```
119+
The C++ function `torch::from_blob` will create an input tensor from the pixel buffer. Note that the shpae of the tensor is `{1,3,224,224}` which represents `NxCxWxH` as we discuessed in above section.
107120
108121
```cpp
109-
auto outputTensor = _impl.forward({inputTensor}).toTensor();
122+
torch::autograd::AutoGradMode guard(false);
123+
at::AutoNonVariableTypeMode non_var_type_mode(true);
124+
```
125+
The above two lines tells the PyTorch engine to do inference only. This is beacuse By default, PyTorch has built-in support for doing auto-differentiation, which is also known as autograd. Since we don't do training on mobile, we can just disable the autograd mode.
126+
127+
Finally, we can call this `forward` function to get the output tensor as the results.
128+
129+
```cpp
130+
auto outputTensor = _impl.forward({tensor}).toTensor();
110131
```
111132
112133
### Collect results
@@ -120,9 +141,7 @@ let sortedResults = zippedResults.sorted { $0.1.floatValue > $1.1.floatValue }.p
120141

121142
### PyTorch demo app
122143

123-
For more complex use cases, we recommend to check out the PyTorch demo application.
124-
125-
The demo app contains two showcases. A camera app that runs a quantized model to predict the images coming from device’s rear-facing camera in real time. And a text-based app that uses a self-trained NLP model to predict the topic from the input string.
144+
For more complex use cases, we recommend to check out the PyTorch demo application. The demo app contains two showcases. A camera app that runs a quantized model to predict the images coming from device’s rear-facing camera in real time. And a text-based app that uses a self-trained NLP model to predict the topic from the input string.
126145

127146
## Build PyTorch iOS libraries from source
128147

0 commit comments

Comments
 (0)