Introduction
High-performance audio on tablets is something we don’t often think about as developers—but we should. Tablets are used as complete portable entertainment systems, and whether your users consciously realize it, audio is a huge component in making entertainment content compelling and engaging.
With Android*-powered devices now representing over half of the tablet market share, Google’s OS is becoming the platform of choice for not just portable mobile entertainment, but entertainment in general. Unfortunately, the Android hardware landscape is populated with a huge range of devices, and not all of them are up to the task of delivering a smooth and compelling entertainment experience.
The three main concerns when it comes to consuming high-performance audio are:
- CPU performance – without a powerful processor the audio experience is likely to be “laggy” or “choppy”
- Battery life – audio processing, as mentioned above, can be CPU intensive, and long battery life is always a requirement for mobile entertainment
- Audio quality – sound needs to be clear, loud, with individual sounds well-defined
By selecting the Intel® Atom™ processor-powered Samsung Galaxy Tab* 3 10.1, which consistently benchmarks at the same level or better to products powered by other processor architectures, performance isn’t an issue. The combination of high clock speed and dual cores enables flawless audio playback even when multitasking.
Additionally, the Intel Atom SOCs used in Android devices are optimized for power consumption, and as a result they have very strong battery life, which the Galaxy Tab 3 maximizes.
The final element is audio quality, and this is provided by the integrated Dolby Digital Plus* audio hardware. The Dolby hardware provides several key features, most notably being the Volume Maximizer, which boosts output without causing distortion, and the Audio Optimizer, which noticeably improves definition and clarity in music via Dolby’s proprietary audio processing algorithms.
It’s also worth noting, since the Dolby audio solution is a hardware codec, it doesn’t place any additional stress on the CPU so it’s almost “free” to enable. It’s not going to consume substantial additional hardware resources when in use.
Although audio “quality” can be subjective, especially when it comes to enhancing pre-recorded content, it’s clear that Dolby has a considerable technology stack in place with the Dolby Digital Plus codec, and it’s worth exploring some of these technologies.
Dolby Digital Plus Audio Codec Technology
The Dolby Digital Plus audio codec consists of a number of different audio processing technologies that will act on the audio content being played on an enabled mobile device, with the ultimate goal of improving the listening experience. Several key technologies for music playback are worth pointing out.
Volume Leveler
The Volume Leveler continuously monitors or “listens” to the audio being played by using what Dolby refers to as a “psychoacoustic model of loudness perception” that gauges at what loudness a human will perceive the audio. This value is used to dynamically adjust the volume level to give a perceived consistent loudness. There’s an additional technology called “Auditory Scene Analysis” that is used in conjunction with the leveler to make sure the audio is not adjusted improperly, specifically so it doesn’t over amplify intentionally quiet sounds or moments in the music.
Under the hood, Dolby states the Volume Leveler is also able to target both individual audio channels and/or individual frequency bands to prevent a perceived “pumping” or “breathing” effect when altering the audio.
Volume Maximizer
One of the biggest benefits the Dolby Digital Plus codec provides is Volume Maximizing, which aims to bring consistently loud output through the typical small speakers found on the majority of mobile devices. After the Volume Leveler has processed the audio, the Maximizer increases the amplitude of the sound by up to a very noticeable 12dB.
To avoid over-amplifying the signal, which could cause clipping, the Volume Maximizer uses a “look-ahead limiter.” The limiter combines the increased signal gain with multiband compression to maximize volume without causing distortion.
Audio Optimizer
The frequency response of the speakers included in mobile devices is typically less than ideal. In other words, the output level can change substantially for different frequencies. This can give an unnatural or muffled sound when listening to music. The Audio Optimizer uses a set of filters to try to correct any irregularities in the speakers themselves. These filters are customized for each mobile device model. The end goal is to deliver natural balanced audio with a perceived flat frequency response, which on most devices, increases bass and treble response.
Audio Regulator
The final step in the audio processing that we’re interested in as music listeners, is the Audio Regulator. Boosting volume output on mobile devices can introduce acoustic distortion. This can come from the amplifiers themselves, the speakers being overdriven, or resonation and “rattling” in the case of the phone or tablet. This behavior is very specific to the individual device and can vary for different frequencies.
The Regulator applies multiband compression that is again tuned to the specific mobile device. Both the Audio Regulator and the Audio Optimizer are calibrated by Dolby so the specific processing performed is unique to the specific device type.
Additionally, constraints are applied to the compression to maintain the timbre of the audio.
Finally, the regulator is volume dependent. So distortion control can be reduced or turned off at lower volume levels.
Why is Audio important on Mobile?
Tablets and smartphones are becoming the new preferred devices for watching and consuming entertainment content. It’s not easy to convey just how much of an impact strong sound quality can make on the overall experience. Recently, a study by Parks Associates (commissioned by Dolby) attempted to measure the importance of good audio. The study’s results showed that the majority of smartphone owners do consider sound quality to some extent when choosing a smartphone or tablet.
However, after hearing an audio demonstration from a device that features enhanced and improved sound reproduction, the majority of users surveyed acknowledged that sound quality should be an important feature for their mobile devices.
A simple demonstration of Dolby Digital’s audio technology will likely make you agree.
Using the Dolby Audio API in Android applications
Dolby has made it straightforward to enhance audio on Android devices that include Dolby Digital Plus hardware. The Dolby Developer portal contains a free API to enable Dolby audio processing in traditional Java* Android applications as well as supporting some 3rd party frameworks including Xamarin, Unity 3D*, and Cordova*.
Because the Dolby Digital solution works at a hardware level, it’s able to improve audio regardless of how the audio content is being loaded and played within the source code.
To enable Dolby Digital audio processing you’ll need to follow a few steps:
- Download and install the Dolby Digital API in your mobile application from https://developer.dolby.com
- Get an instance of the DolbyAudioProcessing object. This is quite easy; simply call GetDolbyAudioProcessing() against the Dolby API.
- Create a Dolby listener object. Typically you’ll make your current Activity the Dolby listener object to ease the implementation, but this can be a separate class as well.
A Java code snippet is shown below:
import com.dolby.dap.DolbyAudioProcessing;
import com.dolby.dap.OnDolbyAudioProcessingEventListener;
import com.dolby.dap.DolbyAudioProcessing.PROFILE;
public <strong>class</strong> MainActivity
extends Activity implements OnDolbyAudioProcessingEventListener
{
DolbyAudioProcessing mDolbyAudioProcessing;
...
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
...
mDolbyAudioProcessing = DolbyAudioProcessing.getDolbyAudioProcessing(this, DolbyAudioProcessing.PROFILE.GAME, this);
if (mDolbyAudioProcessing == null){
Toast.makeText(this, "Dolby isn't available on this device", Toast.LENGTH_SHORT).show();
finish();
return;
}
}
...
}Case Study
I had the opportunity to create a music streaming application to help indie artists gain exposure and showcase their music. Having a device to showcase both the application and the music was also paramount. So for this project the x86-powered Galaxy Tab 3 10.1 was chosen for performance, battery life, and sound quality.
The music content is streamed from SoundCloud and played back using the MediaPlayer object included in the Android API.
To allow for code re-use across other platforms (WPF, Windows* Store, etc.), I used a 3rd party development tool, Xamarin Android, which allows native Android applications to be written in .NET C#. However, because Xamarin leverages the native Android Java APIs, the development steps are essentially the same as when writing Java applications.
Real-world Dolby API example
For my Android Activity responsible for playing music tracks, I decided to make it the listener for the Dolby Digital API, which means it’s responsible for listening and responding to the Dolby API through methods defined in IOnDolbyAudioProcessingEventListener. I also made the Activity implement the Media Player’s IOnCompletionListener interface making it the singular point for audio playback. This approach works well, but you can separate out these interfaces into separate classes as your application architecture requires.
As a result, my class signature looks like this:
public class Activity1 : Activity, MediaPlayer.IOnCompletionListener, IOnDolbyAudioProcessingEventListener
I kept instances of both the MediaPlayer object accessible as class level variables. Note, the DoblyAudioProcessing object should be made static, and you should never hold more than one instance of this object within an application. If you need to access the DolbyAudioProcessing object across Activities, it should be shared via a singleton or some type of global reference.
To initialize Dolby audio processing, I created an InitAudio() helper method to safely retrieve a reference to the DolbyAudioProcessing object. Since compatibility with other devices is still necessary, this method fails silently on devices that don’t include the Dolby codec.
When requesting the processing object, I also set the Dolby Digital processing mode. In this case I selected music, which is designed to improve musical clarity, balance, and perceived loudness. The audio improvements are equally impressive using the tablet’s built-in speakers, ear buds, or a line out to external audio hardware.
void InitAudio()
{
if (mDolbyAudioProcessing != null)
return;
mDolbyAudioProcessing = DolbyAudioProcessing.GetDolbyAudioProcessing(this, DolbyAudioProcessing.PROFILE.Music, this);
if (mDolbyAudioProcessing == null)
{
Toast.MakeText(this, "Dolby Audio Processing load failed", ToastLength.Short).Show();
return;
}
}Finally, I check and save the Dolby processing state when the Activity starts and restore the state when our application executes in the background or is closed. Because the Dolby Digital hardware affects all audio running on the device, this is an important step and can be accomplished easily via the Activity’s lifecycle methods onResume() and onPause().
SoundCloud API
The SoundCloud API allows you to build applications that take advantage of the full functionality available on the SoundCloud website. HTTP requests are made to a set of endpoint URLs to request information and perform actions.
Working with the SoundCloud API is well documented, but because the API itself is quite deep, it can initially be intimidating. The first step is to take a look at the REST documentation: https://developers.soundcloud.com.
The easiest way to get started, is to use the Java API wrapper available on GitHub: https://github.com/soundcloud/java-api-wrapper; however, I’ll go through the steps of incorporating SoundCloud a bit more manually.
You need to create a SoundCloud client id, which is a unique ID for your application. You’ll include this when interacting with the REST APIs for accessing publicly accessible content.
Playing SoundCloud Content
Let’s walk through the process of accessing a specific artist on SoundCloud, downloading album information, reading the individual tracks, and then playing a song. To do this we’ll be interacting with SoundCloud’s RESTful service as well as decoding the returned JSON data.
Step 1: Create a Client ID
As mentioned above, this ID is free and can be created here: https://soundcloud.com/you/apps
It will look something like this: “7de8bc189e5ba2ab12fa4223951fabb8”
Step 2: Find the Artist’s ID
You’ll need to find the user ID of the artist you wish to play, which can be found directly on the SoundCloud website or by searching via the SoundCloud API. Since we’re just learning the API, I’ll share a quick way of finding the ID by hand: Open any artist page directly on the SoundCloud site, click Share, and then Embed. In the Code & Preview box you’ll see an iFrame tag. Within it you should see a link that looks something like this: https%3A//api.soundcloud.com/users/547647. The number is the artist ID you’re looking for.
Step 3: Build a Request to download Albums
We’re now going to build our first request against the SoundCloud API. We’ll start with the address of the API and then build up the full URL using the artist ID from step 2 and our own client ID from step 1.
string request = "http://api.soundcloud.com/" + "users/" + artistID + "/playlists.json?client_id=" + clientID;
Step 4: Send the request
Now simply send the request. Since my application was written in C#, the code looks like this:
public async Task<string> GetUserPlaylists ()
{
HttpClient client = new HttpClient ();
var request = "http://api.soundcloud.com/" + "users/" + artistID + "/playlists.json?client_id=" + clientID;
HttpResponseMessage response = await client.GetAsync (request);
return await response.Content.ReadAsStringAsync ();
}
A quick note: this code will work on any modern .NET platform that includes Microsoft’s HttpClient including WPF, Windows Store, or Silverlight* apps.
Step 5: Deserialize the Object
You can use your favorite JSON library here. When I write .NET code, I use the Newtonsoft Json library.
_setLists = Newtonsoft.Json.JsonConvert.DeserializeObject<List<AlbumData>>(response);
The response variable is the result from the GetUserPlaylists() method in the previous step. This will give us a List of AlbumData objects (defined below).
Step 6: Create the song URL
You’ll likely want to present the collection of Albums to your users to make a selection via a ListView; I won’t go into the details of creating the UI here.
Below is the code for the AlbumData data transfer object. You’ll notice at the bottom it contains a list of TrackData objects representing the individual songs.
Fortunately for us, the data used to populate the TrackData object contains a URL for playback.
Step 7: Play a song
Finally, playing audio from a remote URL via HTTP streaming on Android looks like this:
String url = myTrack.StreamURL; MediaPlayer mediaPlayer = new MediaPlayer(); mediaPlayer.SetAudioStreamType(AudioManager.STREAM_MUSIC); mediaPlayer.SetDataSource(url); mediaPlayer.Prepare(); mediaPlayer.Start();
Where myTrack is the specific TrackData instance for the song you want to play.
SoundCloud Data Transfer Objects
At this point I would like to show the AlbumData and TrackData objects. Although the SoundCloud API is fairly easy to use, unless you want to write a lot of manual JSON parsing code, we’re going to want to use some Data Transfer Objects (or DTOs) to store the parsed JSON data.
For the purposes of browsing albums and playing songs we’re going to need two DTOs: one to represent album data, and a second for the individual songs/tracks. I’ve included my two DTOs below in C#. These are complete and can be used in your own projects.
If you’re looking to go deeper into the SoundCloud APIs, you’re likely going to create additional DTOs.
AlbumData.cs
using Newtonsoft.Json;
using System.Collections.Generic;
namespace SoundCloud.Json
{
public class AlbumData
{
[JsonProperty("id")]
public string Id { get; internal set; }
[JsonProperty("created_at")]
public string CreatedAt { get; internal set; }
[JsonProperty("user_id")]
public string UserId { get; internal set; }
[JsonProperty("user")]
public UserData User { get; internal set; }
[JsonProperty("title")]
public string Title { get; internal set; }
[JsonProperty("permalink")]
public string Permalink { get; internal set; }
[JsonProperty("permalink_url")]
public string PermalinkUrl { get; internal set; }
[JsonProperty("uri")]
public string Uri { get; internal set; }
[JsonProperty("sharing")]
public string Sharing { get; internal set; }
[JsonProperty("embeddable_by")]
public string EmbeddableBy { get; internal set; }
[JsonProperty("purchase_url")]
public string PurchaseUrl { get; internal set; }
[JsonProperty("artwork_url")]
public string ArtworkUrl { get; internal set; }
[JsonProperty("description")]
public string Description { get; internal set; }
[JsonProperty("label")]
public UserData Label { get; internal set; }
[JsonProperty("duration")]
public int Duration { get; internal set; }
[JsonProperty("genre")]
public string Genre { get; internal set; }
[JsonProperty("tag_list")]
public string TagList { get; internal set; }
[JsonProperty("label_id")]
public string LabelId { get; internal set; }
[JsonProperty("label_name")]
public string LabelName { get; internal set; }
[JsonProperty("streamable")]
public bool Streamable { get; internal set; }
[JsonProperty("downloadable")]
public bool Downloadable { get; internal set; }
[JsonProperty("tracks")]
public List<TrackData> Tracks { get; internal set; }
}
}TrackData.cs
using System;
using Newtonsoft.Json;
namespace SoundCloud.Json
{
[JsonObject]
public class TrackData
{
[JsonProperty("id")]
public long Id { get; internal set; }
[JsonProperty("created_at")]
public DateTime CreatedAt { get; internal set; }
[JsonProperty("user_id")]
public long UserId { get; internal set; }
[JsonProperty("duration")]
public long Duration { get; internal set; }
[JsonProperty("commentable")]
public bool Commentable { get; internal set; }
[JsonProperty("state")]
public string State { get; internal set; }
[JsonProperty("sharing")]
public string Sharing { get; internal set; }
[JsonProperty("tag_list")]
public string TagList { get; internal set; }
[JsonProperty("permalink")]
public string Permalink { get; internal set; }
[JsonProperty("description")]
public string Description { get; internal set; }
[JsonProperty("streamable")]
public bool Streamable { get; internal set; }
[JsonProperty("downloadable")]
public bool Downloadable { get; internal set; }
[JsonProperty("genre")]
public string Genre { get; internal set; }
[JsonProperty("release")]
public string Release { get; internal set; }
[JsonProperty("purchase_url")]
public string PurchaseUrl { get; internal set; }
[JsonProperty("label_id")]
public string LabelId { get; internal set; }
[JsonProperty("label_name")]
public string LabelName { get; internal set; }
[JsonProperty("isrc")]
public string ISRC { get; internal set; }
[JsonProperty("video_url")]
public string VideoUrl { get; internal set; }
[JsonProperty("track_type")]
public string TrackType { get; internal set; }
[JsonProperty("key_signature")]
public string KeySignature { get; internal set; }
[JsonProperty("bpm")]
public long? BPM { get; internal set; }
[JsonProperty("title")]
public string Title { get; internal set; }
[JsonProperty("release_year")]
public string ReleaseYear { get; internal set; }
[JsonProperty("release_month")]
public string ReleaseMonth { get; internal set; }
[JsonProperty("Release_day")]
public string ReleaseDay { get; internal set; }
[JsonProperty("original_format")]
public string OriginalFormat { get; internal set; }
[JsonProperty("license")]
public string License { get; internal set; }
[JsonProperty("uri")]
public string Uri { get; internal set; }
[JsonProperty("permalink_url")]
public string PermalinkUrl { get; internal set; }
[JsonProperty("artwork_url")]
public string ArtworkUrl { get; internal set; }
[JsonProperty("waveform_url")]
public string WaveformUrl { get; internal set; }
[JsonProperty("user")]
public UserItemData User { get; internal set; }
[JsonProperty("stream_url")]
public string StreamUrl { get; internal set; }
[JsonProperty("download_url")]
public string DownloadUrl { get; internal set; }
[JsonProperty("playback_count")]
public long PlaybackCount { get; internal set; }
[JsonProperty("download_count")]
public long DownloadCount { get; internal set; }
[JsonProperty("favoritings_count")]
public long FavoritingsCount { get; internal set; }
[JsonProperty("comment_count")]
public long CommentCount { get; internal set; }
[JsonProperty("attachments_uri")]
public string AttachmentsUri { get; internal set; }
}
}Conclusion and Final Thoughts
SoundCloud
The information above should give you a solid start to using SoundCloud. But if you need a deeper implementation, i.e., uploading or managing content, you’ll need to create additional Data Transfer Objects and be careful to ensure the data in your application is kept in sync with SoundCloud. If you’re a Java developer, you also may want to look at the Android Sharing Kit: https://github.com/soundcloud/android-intent-sharing/wiki
Dolby Digital with SoundCloud
A common problem with streaming audio services is the inconsistent leveling across sound files. Although this doesn’t have anything to do with how the service provides the content, it was a great added benefit to have the Dolby Digital API normalize the volume. Inconsistent leveling becomes very noticeable when switching between songs from different artists.
Android MediaPlayer
The MediaPlayer object in the Android API is a convenient way for playing local or streaming audio content, but it can be “temperamental.” Although you should always test on a wide range of devices when writing Android applications, it is especially true when using the MediaPlayer object. You can’t assume it will “just work” on all devices, even if you’re seeing ideal behavior when testing. Especially when it comes to devices with lower specifications. Consider using the Alpha & Beta testing features of the Google Play store or use a testing service that will allow you to run your app on wide range of handsets and tablets.
Performance
As mentioned above, be careful with the MediaPlayer as it might not behave as expected on lower performance devices. Also ensure you take the time to optimize other parts of your application to keep your audio playing without interruption. Always run code that waits on web services off the UI thread and make sure to take advantage of things like cell reuse and View Holders in your List Controls to reduce memory usage and maximize performance. See the Making ListView Scrolling Smooth documentation for some great optimization tips: http://developer.android.com/training/improving-layouts/smooth-scrolling.html
Devices
The Samsung Galaxy Tab 3 10.1turned out to be a fantastic presentation device, and I’m still impressed by the sound improvements the Dolby Digital hardware can make with streamed content. Even though it’s not available on all devices, it still makes sense to take the time to integrate the Dolby Digital API into your Android applications. It’s impressively quick to implement and makes a drastic improvement in audio output and clarity.
Moving Forward
The Samsung Galaxy Tab 3 10.1 still competes well in the marketplace, but mobile moves fast. It’s exciting to see new Intel Atom quad core processor-powered tablets coming to market. The Dell Venue* 8 7000 is looking to be a class-leading device when it’s released in Q4 2014. In addition, we’re seeing strong adoption of the Dolby Digital Plus hardware; one notable hardware manufacturer being Lenovo, which is releasing several tablets that include the Dolby codec. I’ve just recently switched over to the Lenovo S8-50f as my primary Android development device. It is powered by the Intel Atom chipset Z3745 and comes with quad-core x86 power, 2 gigabytes of ram, and of course, Dolby Digital Plus.
About the Author
Adrian Stevens has over 14 years of experience developing mobile applications; specializing in C# and C++ cross-platform development. Adrian’s expertise includes user interface architecture, audio/signal processing, sensors, and mathematics. Based out of Vancouver, Canada, Adrian has a passion for technology, and entrepreneurial spirit, and co-runs a C# cross-platform development Meetup community.
Adrian first began developing mobile applications in 2001 for the Palm Pilot and Pocket PC mobile platforms and has successfully founded and led a successful boutique mobile development company. More recently, Adrian has become a software development instructor for mobile and cross-platform application architecture and development.