Over the years, the photosensitive element has been moving towards reducing the size and strengthening the signal processing, but Sony has opened a new path on the 12.3MP photosensitive elements of the IMX500 and IMX501 released today, focusing on the built-in simple AI and machine learning functions. The concept proposed by Sony here is to directly encapsulate the AI-processed logic chip and the photoreceptor element. Therefore, in addition to outputting the picture. For example, if a machine learning model for object recognition is placed in the chip, the chip can output the text "dog" when the appears on the screen, not necessarily the image.
Compared with the existing common AI mode- the photosensitive processor, the cloud performs the comparison and operation of the machine learning model and then returns to the general-purpose processor-compared to Sony. The chip eliminates almost all the steps in the middle, so it not only does not need to worry about how to connect to the cloud but also maintains the privacy of users. For example, if this photoreceptor is used on a surveillance recorder at home, if the purpose is only to push notifications when a dog enters the screen, the photoreceptor can directly send only the text "dog" to the processor, not only there is no screen upload to upload at all, and even sending the screen to the processor can be saved.
It can be equipped with different models, which also means that the same hardware can be used in many different places as required. An example from Sony is that in the store, the surveillance video camera at the store can automatically estimate the stock of goods on the shelf, or start recording when it is recognized that someone has entered the screen to save power and storage space. Of course, this will not completely replace cloud-based machine computing, and there should not be enough horsepower to build machine learning models, but it can greatly enhance the function of simple cameras.
IMX500 (unpackaged version) has been shipped to Sony partners, while IMX501 (packaged version) will be shipped in June. The device will be available from the end of this year to next year, but it should be based on surveillance video recorders, not mobile phones.