live555源码分析（七）播放过程

喜欢ヅ旅行 2024-04-18 21:06 107阅读 0赞

**live555源码分析系列**

[live555源码分析（一）live555初体验][live555_live555]

[live555源码分析（二）基本组件上][live555]

[live555源码分析（三）基本组件下][live555 1]

[live555源码分析（四）RTSPServer分析][live555_RTSPServer]

[live555源码分析（五）DESCRIBE请求的处理][live555_DESCRIBE]

[live555源码分析（六）SETUP和PLAY请求的处理][live555_SETUP_PLAY]

[live555源码分析（七）播放过程][live555 2]

[live555源码分析（八）多播][live555 3]

# live555源码分析（七）播放过程 #

本文衔接上文，分析播放过程，涉及音视频数据如何获取、如何处理以及如何发送

上一篇文章我们分析到`StreamState::startPlaying`函数，本文从这里开始分析，其定义如下

void StreamState
    ::startPlaying(Destinations* dests, unsigned clientSessionId,
    	       TaskFunc* rtcpRRHandler, void* rtcpRRHandlerClientData,
    	       ServerRequestAlternativeByteHandler* serverRequestAlternativeByteHandler,
    	       void* serverRequestAlternativeByteHandlerClientData) {
        
        /* 将目的添加到消费者中 */
        if (dests->isTCP) { //TCP
            fRTPSink->addStreamSocket(dests->tcpSocketNum, dests->rtpChannelId);
            fRTCPInstance->addStreamSocket(dests->tcpSocketNum, dests->rtcpChannelId);
        } else { //UDP
            fRTPgs->addDestination(dests->addr, dests->rtpPort, clientSessionId);
            fRTCPgs->addDestination(dests->addr, dests->rtcpPort, clientSessionId);
        }
        
        /* 开始播放 */
        fRTPSink->startPlaying(*fMediaSource, afterPlayingStreamState, this);
    }

从上面可以看到，会将目的添加到RTPSink中，然后调用`RTPSink`开始播放，并指定好它的源为`fMediaSource`

在我们[live555源码分析（四）RTSPServer分析][live555_RTSPServer]中实现的这个例子中

`RTPSink`是这样产生的

RTPSink* H264VideoFileServerMediaSubsession
    ::createNewRTPSink(Groupsock* rtpGroupsock,
    		   unsigned char rtpPayloadTypeIfDynamic,
    		   FramedSource* /*inputSource*/) {
      return H264VideoRTPSink::createNew(envir(), rtpGroupsock, rtpPayloadTypeIfDynamic);
    }

`H264VideoRTPSink`的作用是将H264的NALU进行RTP打包发送

`fMediaSource`是这样产生的

FramedSource* H264VideoFileServerMediaSubsession::createNewStreamSource(unsigned /*clientSessionId*/, unsigned& estBitrate) {
      estBitrate = 500; // kbps, estimate
    
      ByteStreamFileSource* fileSource = ByteStreamFileSource::createNew(envir(), fFileName);
      if (fileSource == NULL) return NULL;
      fFileSize = fileSource->fileSize();
    
      return H264VideoStreamFramer::createNew(envir(), fileSource);
    }

live555对数据源的类很多采用了装饰者模式，如上述中，`ByteStreamFileSource`用于读取文件字节流，`H264VideoStreamFramer`用于读取h264文件的nalu

搞清楚生产者和消费者对应谁之后，我们继续分析`fRTPSink->startPlaying()`，它的定义如下

Boolean MediaSink::startPlaying(MediaSource& source,
    				afterPlayingFunc* afterFunc,
    				void* afterClientData) {
    	return continuePlaying();
    }

`continuePlaying`在本例中对应如下

Boolean H264or5VideoRTPSink::continuePlaying() {
    	fOurFragmenter = new H264or5Fragmenter(fHNumber, envir(),
                                               fSource, OutPacketBuffer::maxSize,
                                               ourMaxPacketSize() - 12/*RTP hdr size*/);
    
        fSource = fOurFragmenter;
        
        return MultiFramedRTPSink::continuePlaying();
    }

可以看到，这里又会将我们定义的生产者`H264VideoStreamFramer`对象作为参数传递给`H264or5Fragmenter`创建新的生产者，然后修改`fSource`

这里使用的也是装饰者模式，`H264or5Fragmenter`的作用是将NALU进行特定的RTP打包

从头到尾的生产者关系如下

ByteStreamFileSource //文件字节流
    	|
    H264VideoStreamFramer //提取NALU
    	|
    H264or5Fragmenter //对NALU进行RTP打包

到这里，相信你不会觉得定义这么多的生产者很乱，反而会觉得设计得挺巧妙

接着往下看

Boolean MultiFramedRTPSink::continuePlaying() {
    	buildAndSendPacket(True);
    }

void MultiFramedRTPSink::buildAndSendPacket(Boolean isFirstPacket) {
    	unsigned rtpHdr = 0x80000000;
        rtpHdr |= (fRTPPayloadType<<16);
        rtpHdr |= fSeqNo;
        fOutBuf->enqueueWord(rtpHdr);
        
        ...
        packFrame();
    }

`buildAndSendPacket`函数对设置了RTP得头部，其中跳过了时间戳，这个将在稍后设置

RTP头部得定义如下，感兴趣的话可以仔细分析代码

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjQ2MjIwMg_size_16_color_FFFFFF_t_70]

接下来分析`packFrame`函数，其定义如下

void MultiFramedRTPSink::packFrame() {
    	fSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),
        	afterGettingFrame, this, ourHandleClosure, this);
    }

可以看到，上述调用了生产者，获取一帧数据，获取完数据之后会调用`afterGettingFrame`函数

`afterGettingFrame`定义如下，这里请记住，我们稍后分析

void MultiFramedRTPSink::afterGettingFrame(void* clientData, unsigned numBytesRead,
    		    unsigned numTruncatedBytes,
    		    struct timeval presentationTime,
    		    unsigned durationInMicroseconds) {
      MultiFramedRTPSink* sink = (MultiFramedRTPSink*)clientData;
      sink->afterGettingFrame1(numBytesRead, numTruncatedBytes,
    			   presentationTime, durationInMicroseconds);
    }

在此例中`fSource`对应`H264or5Fragmenter`，看一看它的`getNextFrame`函数定义

void H264or5Fragmenter::doGetNextFrame() {
    	if (fNumValidDataBytes == 1) { //缓存没有数据
        	fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,
                                       afterGettingFrame, this,
                                       FramedSource::handleClosure, this);
        } else { //缓存有数据
         	/*
         	 * 打包方式
         	 * 1.单NALU模式
         	 * 2.分片模式
         	 * 3.聚合模式
         	 */
            
            FramedSource::afterGetting(this);
        }
    }

首先，如果缓存里没有数据，那么就会调用`fInputSource`来获取一帧数据，此例中`fInputSource`对应的是`H264VideoStreamFramer`，调用`H264VideoStreamFramer`的`getNextFrame`函数将获取一个NALU

如果缓存里有数据，那么就进行RTP打包，这里的数据是位于RTP的载荷位置，这里实现了三种NALU的RTP打包模式，单NALU打包、分片打包、聚合打包

我们首先来分析`getNextFrame`函数，此例中，其定义如下

void MPEGVideoStreamFramer::doGetNextFrame() {
        fParser->registerReadInterest(fTo, fMaxSize);
        continueReadProcessing();
    }
    
    void MPEGVideoStreamFramer::continueReadProcessing() {
        unsigned acquiredFrameSize = fParser->parse();
    	afterGetting(this);
    }

最终会通过`fParser->parse`来解析获取数据，然后调用`afterGetting`

`fParser`在`H264or5VideoStreamFramer`的构造函数中构造，其作用是解析H264或者H265文件

H264or5VideoStreamFramer
    ::H264or5VideoStreamFramer(int hNumber, UsageEnvironment& env, FramedSource* inputSource,
    			   Boolean createParser, Boolean includeStartCodeInOutput)
    {
        fParser = createParser
            ? new H264or5VideoStreamParser(hNumber, this, inputSource, includeStartCodeInOutput)
    }

好的，在获取完一帧后，会调用`afterGetting`函数，其对应

void H264or5Fragmenter::afterGettingFrame(void* clientData, unsigned frameSize,
    					  unsigned numTruncatedBytes,
    					  struct timeval presentationTime,
    					  unsigned durationInMicroseconds) {
      H264or5Fragmenter* fragmenter = (H264or5Fragmenter*)clientData;
      fragmenter->afterGettingFrame1(frameSize, numTruncatedBytes, presentationTime,
    				 durationInMicroseconds);
    }

void H264or5Fragmenter::afterGettingFrame1(unsigned frameSize,
    					   unsigned numTruncatedBytes,
    					   struct timeval presentationTime,
    					   unsigned durationInMicroseconds) {
        fNumValidDataBytes += frameSize;
        fSaveNumTruncatedBytes = numTruncatedBytes;
        fPresentationTime = presentationTime; //本帧数据的时间戳
        fDurationInMicroseconds = durationInMicroseconds; //下一次获取数据的间隔时间
    
        doGetNextFrame();
    }

可以看到在获取到一帧数据后，又调用了`doGetNextFrame`函数

void H264or5Fragmenter::doGetNextFrame() {
    	if (fNumValidDataBytes == 1) { //缓存没数据
        	fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,
                                       afterGettingFrame, this,
                                       FramedSource::handleClosure, this);
        } else { //缓存有数据
         	/*
         	 * 打包方式
         	 * 1.单NALU模式
         	 * 2.分片模式
         	 * 3.聚合模式
         	 */
            
            FramedSource::afterGetting(this);
        }
    }

第二次调用这个函数，现在会进入else分支，对数据进行RTP封装，然后调用`FramedSource::afterGetting`

如果对于RTP封装感兴趣的话，可以看[从零开始写一个RTSP服务器（三）RTP传输H.264][RTSP_RTP_H.264]

我们接着查看`FramedSource::afterGetting`，其定义在下面函数调用中指定

void MultiFramedRTPSink::packFrame() {
    	fSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),
        	afterGettingFrame, this, ourHandleClosure, this);
    }

定义如下

void MultiFramedRTPSink::afterGettingFrame1(unsigned frameSize, unsigned numTruncatedBytes,
    		     struct timeval presentationTime,
    		     unsigned durationInMicroseconds) {
        ...
        
    	doSpecialFrameHandling(curFragmentationOffset, frameStart,
                               numFrameBytesToUse, presentationTime,
                               overflowBytes);
        
        sendPacketIfNecessary();
    	...
    }

这里忽略了许多东西，只保留了其中最主要的部分

首先调用`doSpecialFrameHandling`对RTP包进行处理，本例中，其定义如下

void H264or5VideoRTPSink::doSpecialFrameHandling(unsigned /*fragmentationOffset*/,
    						 unsigned char* /*frameStart*/,
    						 unsigned /*numBytesInFrame*/,
    						 struct timeval framePresentationTime,
    						 unsigned /*numRemainingBytes*/) {
    	...
    	if()
        	setMarkerBit();
        
        setTimestamp(framePresentationTime);
    }

可以看到此函数会设置默写标志位和时间戳

到了这里，一个RTP包就完成了，接下来看`sendPacketIfNecessary`，其定义如下

void MultiFramedRTPSink::sendPacketIfNecessary() {
    	...
        fRTPInterface.sendPacket(fOutBuf->packet(), fOutBuf->curPacketSize()
        
    	...
    	envir().taskScheduler().scheduleDelayedTask(uSecondsToGo, (TaskFunc*)sendNext, this);
    }

首先将数据传送给所有的客户端，然后添加一个定时任务，准备下一次发送

整个过程就是这样，确实有点绕，我将其画为流程图，如下

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjQ2MjIwMg_size_16_color_FFFFFF_t_70 1]

本文到这里就结束了

[live555_live555]: https://blog.csdn.net/weixin_42462202/article/details/100626262
[live555]: https://blog.csdn.net/weixin_42462202/article/details/100677974
[live555 1]: https://blog.csdn.net/weixin_42462202/article/details/100709515
[live555_RTSPServer]: https://blog.csdn.net/weixin_42462202/article/details/100711104
[live555_DESCRIBE]: https://blog.csdn.net/weixin_42462202/article/details/100713429
[live555_SETUP_PLAY]: https://blog.csdn.net/weixin_42462202/article/details/100733866
[live555 2]: https://blog.csdn.net/weixin_42462202/article/details/100746999
[live555 3]: https://blog.csdn.net/weixin_42462202/article/details/100748292
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjQ2MjIwMg_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20190809200342276.PNG?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjQ2MjIwMg==,size_16,color_FFFFFF,t_70
[RTSP_RTP_H.264]: https://blog.csdn.net/weixin_42462202/article/details/99089020
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjQ2MjIwMg_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20190911194413496.PNG?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80MjQ2MjIwMg==,size_16,color_FFFFFF,t_70