UC Santa Barbara

This dissertation focuses on the predictive coding of video contents based on optimal prediction principles.

In the first part of the dissertation, the prediction scheme of error-resilient video coding with lossy networks is investigated. With packet-based networks suffering from potential packet loss, the prediction quality may be severely impacted not only by the loss of information, but also by the error propagation through the prediction loop. To account for such influence of lossy channels, the accurate estimation of end-to-end distortion (EED) is crucial for the encoder to perform optimal decisions. Although the recursive per-pixel optimal estimate (ROPE) and the spectral coefficientwise optimal recursive estimate (SCORE) serve as well-know solutions to optimally estimate EED in the pixel domain and the transform domain, their performance is also constrained by the incompatiblity with recent advanced coding tools. As a first step in this dissertation, the SCORE calculations are modified in order to enable the encoder to consider channel losses accurately while being able to maintain the performance improvement due to the variable block sizes. Furthermore, it is recognized that the existing tools are not designed for lossy networks, and thus a novel framework specifically tailored for this situation is proposed with a soft-reset prediction mode. With the accurate EED estimation approach of such mode established, the encoder is able to fine-tune the error propagation and thus achieves a significant performance gain. As another example, we also establish EED estimation recursions for state estimation of wireless sensor networks and proposed an adaptive approach to account for channel errors in Kalman filter, which further proves the significance of EED estimation.

The second part of the dissertation shifts focus to the bi-directional motion compensated prediction in video coding. It is first pointed out that the conventional scheme of bi-directional motion compensation is sub-optimal, since the existing motion information among the reference frames are not efficiently utilized and the block-based motion estimation is overly crude. To overcome this issue, a novel framework with the co-located reference frame (CLRF) is proposed, where a reference frame (CLRF) is interpolated by the motion field estimated between the reference frames at the decoder, without explicit transmission of such motion field. An extra step of block-based motion estimation is then performed on top of CLRF to correct possible motion offsets. Performance gains shown by experimental results prove the effectiveness of the framework.

Estimating motion field at the decoder, however, suffers from quantization error and significant complexity rise. Therefore, we then propose to apply an estimation-theory based approach to utilize the motion vectors (MVs) already available to the decoder, and treat the associated reference pixels as observations of the current block. An optimal linear estimator is then derived and used to interpolate the CLRF. With greatly reduced complexity, this approach also provides significant coding performance improvement.

The available MV candidates are then also utilized to predict the MV of the current block, in order to remove redundancy when coding MVs. Instead of a linear estimator, a novel scheme is designed to find the MV prediction that is most consistent with the MV candidates (observations) given a certain pixel correlation model. Experimental results show that the proposed scheme also achieves a boost in coding performance.