Signal Processing- Behind the multimedia world.
Our modern life takes place in a world that is full of images, sounds and videos. You probably wake up by listening to some relaxing tune on your phone,send a voice message to say happy birthday to a close relatives, listen to some music while mediation, and Then, you receives a notification to check the cool photos your friend has just sent you from yesterday’s party and, while you’re checking the news, a trailer from a best movie appears on your screen. It’s still early in the morning and you’ve already experienced the digital world. The day, however, hasn’t even started and you’ll experience lot of photos, more movies,huge music and documents that will be transferred through a data network to one or several of your electronic devices.
signal processing is the mathematical tool that lets us enjoy our world transparently and naturally . We could even say signal processing is the a science that lets us capture the information we hear and see for being later stored, sent and reproduced as many times as we like through our laptops, tablets,or smartphones . However, when it comes multimedia signal processing isn’t only about storing, reproducing or sending content. It’s also at the content creation stage itself, helping videographers , musicians and photographers to develop their creativity through the use of digital tools that use signal processing to manipulate multimedia particulars in a meaningful way.
Data Compression
One of the challenges that signal processing faces with multimedia applications is that audiovisual signals carry a lot of content, specially when they are digitally acquired at a very high quality. Analog-to-Digital conversion is the process that converts the signal into digital form. translating sounds and images to numbers by means of an appropriate sensor and conversor. The higher the amount of numbers we take, the higher the quality of the reproduced signal, but such quality comes at the expense of producing a huge amount of data. Storing and sending such a huge amount of data isn’t usually a good way, since the memory, processing powers and communication bandwidth of digital devices is always limited. Signal processing for data compression is aimed toward reducing the size of the data to allow the applications to process information without unnecessarily wasting resources and letting users experience multimedia content without annoying delays. Data compression allows us to store hundreds of music songs, photos and videos in our devices. Additionally, it helps to accelerate the media content present in most websites and to experience music and videos in real-time from content providers. digital television or video streaming in YouTube we should thanks to the advances of signal processing in the data compression fields.
Data compression can be achieved either without any loss of information (lossless compression) or by permitting some information loss (lossy compression). In lossless formats, the information recovered from the compressed data is exactly the same as the raw original data. However, the reduction that can be achieved with lossless formats is usually quite limited. In contrast, lossy compression techniques reduce the size of audio, photos and videos by discarding information in such a way that humans cannot easily perceive the difference between the original and the compressed data. The good thing is that the reduction rate obtained by lossy formats is high and the data can be easily adapted to a range of applications and devices. But, how does signal processing distinguish between perceptually relevant and irrelevant data?
Audio Compression
When the data comes from a sound signal such as music, signal processing achieves data compression by following a sophisticated process where the sound data is analyzed in terms of the frequencies that make up the sound. The human hearing system is not equally sensitive to all the audible frequencies: that’s why two sounds having the same power are not perceived equally loud, especially if one of them has a very low or a very high pitch. Moreover, the interaction among the frequencies making up a given sound in our hearing system usually leads to masking effects, which make some of these frequencies inaudible in the presence of others. Perceptual audio coding takes advantage of this fact to reduce the size of audio data by discarding information corresponding to irrelevant frequencies that are being masked. Therefore, even though we’re losing information from our raw sound data, this process is performed in a smart way, making the compressed sound almost indistinguishable from the original. Signal processing provides the mathematical framework necessary to perform this process as efficiently as possible.
Digital audio streams in most current multimedia technologies and broadcasting formats make use of audio coding standards that are based on the above principles, such as the well-known Advanced Audio Coding (AAC), which is the default audio standard for YouTube, iPhone, or PlayStation.
Image and Video Compression
Image compression techniques also make use of the peculiarities of human perception to select the information that must be retained for compressing images without losing too much quality. In this case, signal processing also provides a framework for analyzing images in terms of the spatial frequencies that make up small areas of an image. Since the pixels corresponding to small areas tend to be quite similar, it’s possible to eliminate the fine spatial variations that the human visual system would hardly perceive. As a result of this process, information corresponding to very fine spatial detail can be selectively discarded without affecting too much the quality of the compressed version of the image. Moreover, the accuracy of the human eye in detecting detail for the brightness component of an image is higher than for the color information, a fact that is also used to reduce the amount of data in color images. Signal processing again, provides the mathematical tools for carrying out this analysis process, allowing our everyday applications to manage digital images comfortably and efficiently.
Lossy image compression is used in popular image formats, such as JPEG, which is broadly used to store photos in our digital cameras and to send images throughout the Internet.
You are probably now wondering how does signal processing reduce the data rate of videos. Of course, signal processing also provides methods to allow applications manipulate video information. A video is basically a set of images (or frames) shown rapidly one after another to create motion. The same principles discussed for image compression can be used to compress video information, but in this case, signal processing also exploits the fact that the small areas making up one frame are very similar to the ones of its neighboring frames. Fortunately, the changes from one frame to the next can be efficiently predicted and, combined with the above static image compression techniques, video coding standards are used all the time to store, send and receive video information in our electronic devices.
Digital video coding standards such as MPEG-2 Part 2 or H.264 are used in high-definition digital broadcast television, Blu-rays, Satellite TV and internet streaming applications such as YouTube or Vimeo.
Conclusion
We are immersed in a multimedia world, surrounded by a combination of sounds, images and videos that are an important part of our modern digital life. Signal processing has been a fundamental enabling technology for making all of our favorite gadgets a reality, providing methods for creating, storing, transforming, sending and receiving multimedia information. However, despite the advances in this field, our immersive multimedia world has just begun, and signal processing scientists are currently working on the future of multimedia, developing suitable frameworks for creating new and better immersive environments dealing with virtual and augmented reality, first-person point of view multimedia systems or free viewpoint television.
References: https://en.wikipedia.org/wiki/Data_compression#Audio,https://www.sciencedirect.com/video Compression - an overview | ScienceDirect Topics.
Contributors
Swati More, Mayuresh Patil, Suraj Patil, Vedika Patil.
