Streamezzo S.A. - All rights reserved
Copyright 2001-2010
Audio / Video - Introduction
This article provides you a guided tour to become familiar with manipulation of Audio and Video contents within a Rich Media Application with Streamezzo Rich Media Solution.
You will find here an overview of all manipulated concepts, and also
all available references and resources that will make you comfortable
for developing your own appealing rich media applications, such as
audio players, radio, video players, mobile tv...
Here you will be introduced with the keyaspects of audio / video within rich media:
- Support, i.e. the audio / video formats and encodings
- Streaming, i.e. the progressive download of audio / video content
- Display, i.e. the capabilities to transform a video area
- Controls, i.e. the capabilities to drive audio / video playback
- Volume, i.e. the capabilities to act on sound volume manage or output
- Time, i.e. the related time information
- Events / Errors, i.e. the related events (some are errors) that can raise and be managed
- Components, i.e. the available components (provided by Streamezzo) related to audio / video
In the following sections, when referring to video it means both visual and audio contents; this means when you want to play a video with sound, you don't have to manage separately video and audio: video comes with sound.
Support
For the playback of audio / video content, a Streamezzo-enabled application relies on various solutions, depending on addressed platform (i.e. native or J2ME):
- On native devices
- You can rely on Streamezzo's own 3GPP compliant audio / video player, embedded within the deployed rich media application.
- In addition and complementarily to this, audio / video playback can be delegated to the native audio / video player available on the device.
- Last, also in addition and complementarily to this, audio / video playback can be delegated to a 3rd party audio / video player if any is available on the device.
- On J2ME devices, one can only rely on the corresponding JSR (135) that will handle audio / video playback. As the JSR implementation can usually vary from a device to another, we recommend to check if the device will properly supported the expected media.
The following support information deals with 3GPP compliant audio / video content playback only.
Indeed, in case you rely on a native or 3rd party audio / video player (on native devices), additional formats can be supported by such player (like MP3...).
Audio
Encapsulated as a .3gp file
Supported encodings:
- AAC_LC
- AMR_NB
Video
Encapsulated as a .3gp file
Supported encodings:
- H.263 baseline
- H.264 baseline
- MP4-Video Simple Profile
Streaming
Audio / video contents can be transported to the device then played by a rich media applications by different ways:
- local files: i.e. audio / video files available in the device filesystem (gallery...)
- downloaded files: i.e. files that can be downloaded when using the rich media playing (this case is then similar to a local file playback)
- streamed media (RTSP or HTTP): i.e. progressively delivered by a dedicated streaming server and consumed by the audio / video player; this is the most common configuration for radio (audio) and mobile tv (video)
- streamed rich media: i.e. audio / video content that are streaming within the rich media content streaming, so delivered by Rich Media Server; such configuration is usually used for demo purpose or on specific networks not supporting RTSP.
In order to use audio / video content in your rich media content, you will have to:
Use RemoteMediaStream with a local path as source to declare a local file / with a remote URL as source to access a RTSP or HTTP streaming server.
Use ThreeGPMediaStream or Full3GPMediaStream to declare audio / video content streamed multiplexed with rich media content (delivered by Rich Media Server). The first instruction will make it possible to consume audio / video data progressively while it is received, the second instruction will make it possible to consume audio / video data only once it has fully been downloaded (which is the only applicable option on some devices).
In addition your can also use cmd://setMediaSource command in order to dynamically set / change the source for an Audio or Video node.
- declare the source (and define an associated streamID)
- use a Audio or Video node referring to this declaration (same streamID)
Use RemoteMediaStream with a local path as source to declare a local file / with a remote URL as source to access a RTSP or HTTP streaming server.
Use ThreeGPMediaStream or Full3GPMediaStream to declare audio / video content streamed multiplexed with rich media content (delivered by Rich Media Server). The first instruction will make it possible to consume audio / video data progressively while it is received, the second instruction will make it possible to consume audio / video data only once it has fully been downloaded (which is the only applicable option on some devices).
In addition your can also use cmd://setMediaSource command in order to dynamically set / change the source for an Audio or Video node.
Display
This section obviously deals with video content only.
It is usually convenient to offer multiple displays for a video content: fullscreen, miniature or adaptation to the current screen size. In order to grant all those display configurations, Streamezzo rich media solution offers the following transformations on video:
- positioning: place the center of the video area on the screen
- scaling: stretch the video area width and/or height
- rotation: rotate the video area (restricted to a 90° multiple)
Concerning video scaling there are 2 possible ways to proceed:
- Apply a distinct ratio on the video area width and height
- Specify a display box with defined width and height, in which the video area will have to fit
To position a video area on screen, use Replace feature on its parent Transform translation.
To scale a video area applying a ratio on its width and height, use Replace feature on Video.scale.
To scale a video area making it fit within a specified display box, use Replace feature on Video.displaybox to specify the box, then Video.boxfit to specify the fit strategy (i.e. how the video area scale must be adapted in order to fit in the specified box).
To rotate a video area, use Replace feature on Video.rotate.
To scale a video area applying a ratio on its width and height, use Replace feature on Video.scale.
To scale a video area making it fit within a specified display box, use Replace feature on Video.displaybox to specify the box, then Video.boxfit to specify the fit strategy (i.e. how the video area scale must be adapted in order to fit in the specified box).
To rotate a video area, use Replace feature on Video.rotate.
Controls
The following controls are granted on audio or video playback:
- Play/PlayAt: start playback from start / current play position, or backwards / forward from current play position (specifying a delay in milliseconds)
- Pause: pause playback on current play position
- Stop: stop playback
Use MediaControl to specify the Audio or Video node to control, and which control to perform on it.
Volume
The following operations are possible on sound management:
- Increase / Decrease current sound volume: the volume can be increased or decreased in the volume range configured for the device
- Mute / Unmute sound
Use cmd://volup and cmd://voldown to respectively increase / decrease the current sound volume.
Use cmd://muteon and cmd://muteoff to respectively mute / unmute the sound.
Use cmd://muteon and cmd://muteoff to respectively mute / unmute the sound.
Time
It is usually convenient to display the current playback time for audio / video content.
As this information is usually displayed as a progress bar, Streamezzo rich media solution provides the ability to enslave an animation (i.e. progress bar or counter or whatever) to the current audio or video content playback time progress.
Obviously such animation would:
- start from its beginning when the media starts to play
- progress while the media continues to play
- pause when the media is paused
- end when the media is stopped
Use MediaTimeListenerRegister to specify the Audio or Video node playback time to listen, and which rich media content it should animate consequently.
Events / Errors
When accessing and controling an audio or video content, especially in case it is accessed remotely, it will go through different intermediate phases, each raising a specific event that can be intercepted and managed (i.e. make the rich media content react accordingly, like displaying such info on screen).
Such events can be informative or related to raised errors (in red below):
- Stopped: the Audio/Video Player component is currently stopped
- PlayAttempted: the Audio/Video Player component has received a play command and is now in a state where it tries to play
- Connecting: (availability is media/platform dependant) the Audio/Video Player component is achieving tasks in order to connect to the media location
- Opening: (availability is media/platform dependant) the Audio/Video Player component is achieving analysis on the media content
- Buffering: (availability is media/platform dependant) the Audio/Video Player is buffering the media content
- Playing: the Audio/Video Player is currently playing the content
- Paused: the Audio/Video Player is currently paused
- EndOfClip: the Audio/Video Player has finished playing the whole media (it has reached the end of the media content)
- Error: an error has been encountered by the Audio/Video Player
- RTSP_4XX: a RFC-2326 Client Error Status-Code is reported
- RTSP_5XX: a RFC-2326 Server Error Status-Code is reported
- RTPTimeOut: a timeout occurred of RTP, probably because of a firewall
- ServerUnreachable: a server cannot be reached or closed the connection (DNS/TCP connect failure/TCP close)
Note that some of those events can be device dependent (i.e. can be raised or not).
Use EventListenerRegister to specify the Audio or Video node to watch, which events to listen to, and which rich media content it should affect consequently.
Components
The following available components related to audio / video can be reused to develop quickly rich media applications requiring such User Interface and facilities:
- Music Player: complete UI for audio playback
- VideoPlayer: complete UI for video playback
- Media Player Control Bar: controls on audio / video content
- Media Time and Progress Bar: time progression related to audio / video playback
You can find those components here.
More...
- Read Language Reference in Workbench Developer for details on the tags and commands mentioned above.
- Streamezzo devices testing reports provide details on features support (like JSR-135 on J2ME devices).
