Optimize first-frame rendering

First frame output time is the duration between when a user joins a channel and when they first see the remote video. A shorter first frame output time reduces perceived wait time by rendering video more quickly.

This guide describes two best practices to reduce video rendering time in Video Calling.

Complete the steps in the SDK Quickstart to build a basic Video Calling app.

To reduce video rendering time, Agora provides the following solutions:

• Preload and initialize before joining the channel
  Complete time-consuming operations ahead of time, such as preloading the channel, configuring the rendering view, and enabling accelerated rendering for audio and video frames.

• Join early, subscribe later
  Join the channel in advance but delay subscribing to the audio and video stream. When the user triggers the join operation, subscribe to the host's stream and begin rendering immediately.

The following table compares both solutions:

Characteristic	Preload and initialize early	Join early, subscribe on demand
Applicable scenarios	Most audio and video use cases	Scenarios with very high requirements for first frame rendering speed
Core implementation	Initialize and configure video settings before joining	Join the channel early without subscribing; subscribe only when needed
Cost	Normal billing	May incur additional channel usage fees

The following figure shows the time to output the first frame before optimization and with each solution:

This section describes the implementation logic for both solutions.

Preload and initialize early

The following figure illustrates the essential steps:

Sequence diagram for implementation

Creating and initializing the Video SDK engine takes time. To reduce first-frame display time, Agora recommends initializing the engine when the module is loaded, not when SDK functions are first called.

info

Initialize the engine only once. Avoid creating and destroying it multiple times.

class AgoraQuickStartActivity : AppCompatActivity() {    private var mRtcEngine: RtcEngine? = null        override fun onCreate(savedInstanceState: Bundle?) {        super.onCreate(savedInstanceState)                // Create and initialize the engine during activity creation        initializeAgoraEngine()    }        private fun initializeAgoraEngine() {        try {            val config = RtcEngineConfig().apply {                mContext = applicationContext                mAppId = "Your App ID"                mChannelProfile = Constants.CHANNEL_PROFILE_LIVE_BROADCASTING                mEventHandler = mRtcEventHandler // You'll need to define this            }                        mRtcEngine = RtcEngine.create(config)        } catch (e: Exception) {            throw RuntimeException("Error initializing RTC engine: ${e.message}")        }    }}

Call enableInstantMediaRendering to reduce the time it takes to render the first video frame and play audio after joining a channel.

• Call this method before joining a channel. Ideally, call it right after engine initialization.
• Both host and audience must call this method to benefit from faster rendering.
• To disable this feature, destroy the engine with release, then reinitialize it.

// Enable accelerated rendering before joining the channelmRtcEngine?.enableInstantMediaRendering()

Use setVideoScenario to optimize performance for your specific use case. The SDK applies strategies tailored to the selected scenario.

For example, for a one-on-one call, use APPLICATION_SCENARIO_1V1.

// Set the video scenariomRtcEngine?.setVideoScenario(Constants.APPLICATION_SCENARIO_1V1)

Joining a channel involves acquiring server resources and establishing a connection. Call preloadChannel to handle resource acquisition early and reduce join time.

• The token, channelId, and uid must match the values used in joinChannel.
• Call preloadChannel as soon as you retrieve the required info.
• Don’t call joinChannel immediately after preloadChannel.

private fun prepareChannelInfo(): Int {    uid = getUid()    channelId = getChannelInfo()    token = getTokenFromServer(channelId, uid)        // Preload the channel    mRtcEngine?.preloadChannel(token, channelId, uid)        return 0 // Return success code, adjust as needed}

Setting the rendering view early ensures the first frame displays properly. If the view is not ready, the first frame might be skipped.

If your app knows the remote user ID (For example, from Signaling), set the view immediately. Otherwise, use the onUserJoined callback.

• Set the remote view early:

  fun onShowChannels(channelId: String, remoteUid: Int) {      val canvas = VideoCanvas(null, VideoCanvas.RENDER_MODE_FIT, remoteUid)      mRtcEngine?.setupRemoteVideo(canvas)  }  fun onEIDUserJoined(uid: Int, elapsed: Int) {      // Already set - no additional setup needed  }

• Set the view when the user joins:

  // Event handler class  private val mRtcEventHandler = object : IRtcEngineEventHandler() {      override fun onUserJoined(uid: Int, elapsed: Int) {          // Forward to UI logic          runOnUiThread {              onEIDUserJoined(uid, elapsed)          }      }  }  fun onEIDUserJoined(uid: Int, elapsed: Int) {      val canvas = VideoCanvas(surfaceView, VideoCanvas.RENDER_MODE_FIT, uid)      mRtcEngine?.setupRemoteVideo(canvas)  }

Use startMediaRenderingTracing to monitor first-frame rendering metrics. Results are reported via onVideoRenderingTracingResult.

info

Call this method when the user initiates joining. For example, on a Join button tap. This gives accurate first-frame timing.

private fun onJoinClicked() {    mRtcEngine?.startMediaRenderingTracing()    mRtcEngine?.joinChannel(token, channelId, uid, options)}

Call joinChannel to enter the channel. To speed up first-frame playback, avoid delays like fetching a token in this method.

If you can’t retrieve a token early, consider using a wildcard token.

private fun prepareChannelInfo(): Int {    uid = getUid()    channelId = getChannelInfo()    token = getTokenFromServer(channelId, uid)    return 0 // Return success code}private fun joinChannel(): Int {    val options = ChannelMediaOptions()    return mRtcEngine?.joinChannel(token, channelId, uid, options) ?: -1}

The SDK runs callbacks like onJoinChannelSuccess on the same thread. If one callback is slow, it can delay others—including rendering events.

info

Don’t block the callback thread with network calls, file I/O, or heavy processing.

• Avoid complex operations in onJoinChannelSuccess.
• Don’t block onUserJoined or other rendering-related callbacks.
• Use background threads for heavy logic.

Join early, subscribe on demand

The following figure illustrates the essential steps:

Sequence diagram for implementation

If you know the host’s user ID before joining the channel, call setupRemoteVideoEx as early as possible to set up the rendering view. This ensures the rendering pipeline is initialized in advance, helping avoid delays in displaying the first decoded frame.

If the host’s user ID is not available beforehand, wait for the onUserJoined callback, then call setupRemoteVideoEx.

val canvas = VideoCanvas(getView(), VideoCanvas.RENDER_MODE_FIT, farNextChannel.remoteUid)mRtcEngine?.setupRemoteVideoEx(canvas, connection)

Joining a channel typically takes the most time before the first video frame appears. For use cases like fast channel switching, delay subscribing to media streams to speed up rendering:

1. Call joinChannelEx to join the channel.
2. In ChannelMediaOptions, set autoSubscribeAudio and autoSubscribeVideo to false.
3. Subscribe manually when the user is ready to view content.

// Define connection and event handlerval connection = RtcConnection(channelId, uid)val eventHandler = mRtcEventHandler// Set channel media optionsval options = ChannelMediaOptions().apply {    channelProfile = Constants.CHANNEL_PROFILE_LIVE_BROADCASTING    clientRoleType = Constants.CLIENT_ROLE_AUDIENCE    autoSubscribeAudio = false    autoSubscribeVideo = false}// Join the channel without subscribingmRtcEngine?.joinChannelEx(token, connection, options, mRtcEventHandler)

When the user chooses to view content:

1. Resume media subscription using muteRemoteVideoStreamEx and muteRemoteAudioStreamEx.
2. Call startMediaRenderingTracingEx to log rendering metrics.
3. The SDK reports results in the onVideoRenderingTracingResult callback, which you can use for performance analysis.

fun switchToChannel() {    // Start video rendering tracing    mRtcEngine?.startMediaRenderingTracingEx(connection)        // Resume remote media subscriptions    mRtcEngine?.muteRemoteVideoStreamEx(remoteUid, false, connection)    mRtcEngine?.muteRemoteAudioStreamEx(remoteUid, false, connection)}

Refer to Slow first-frame rendering of remote video when using the Agora Video SDK.