Android 8.0系统源码分析--Camera processCaptureResult结果回传源码分析_camera result metadata

     相机，从上到下概览一下，真是太大了，上面的APP->Framework->CameraServer->CameraHAL，HAL进程中Pipeline、接各种算法的Node、再往下的ISP、3A、Driver，真是太大了，想把它搞懂真不是个简单的事情。不过我们奔着要把它搞懂的目标，一点点的啃，弄懂一点少一点，我们的功力也在不断的前进中一步步的增强。

     本节，我们就来看一下HAL层一帧处理完成，通过HIDL定义的接口processCaptureResult将数据回传的逻辑。我自己使用的Android8.0的系统源码是通过百度云盘分享的，大家可从Android 8.0系统源码分析--开篇中下载，百度云盘的下载链接和密码都有。

     Camera系统也提供了非常多的dump方式，可以dump metadata、dump buffer，对我们分析问题都有非常大的帮助。

     1、MTK系统中camera sensor的信息会通过kernal中imagesensor.c中的逻辑将所有camera sensor info写入到proc/driver/camera_info文件中，我们可以直接cat proc/driver/camera_info查看camera sensor的详细信息，其中记录了sensor类型，比如IMX386_mipi_raw；记录了sensor支持的分辨率大小，还有一些其他原始信息；

     2、我们可以执行adb shell dumpsys media.camera > camerainfo.txt收集camera metadata信息，大家可以试一下，从这里可以轻易得到很多camera metadata信息；

     3、android-8.0.0\system\media\camera\docs\docs.html文件中定义了AOSP所有的metadata，我们可以直接双击用浏览器打开它，就可以看到所有metadata，当然，每个SOC厂商肯定会自己新增一些，一般metadata的命名都是下划线拼接的，比如android.scaler.cropRegion，它的命名为ANDROID_SCALER_CROP_REGION，我们在HAL进程中就可以通过它去查询当前cropRegion的值。

     4、Google提供的基于Camera API2 Demo地址：android-Camera2Basic（普通的预览拍照功能）、android-Camera2Video（录像功能）。

     HIDL接口的定义有ICameraDevice.hal、ICameraDeviceSession.hal、ICameraDeviceCallback.hal，文件目录在hardware\interfaces\camera\device\xx目录下，xx为版本号，我所下载的源码有1.0和3.2两个版本，我们来看一下3.2版本下ICameraDeviceCallback.hal的定义，源码如下：

package android.hardware.camera.device@3.2;
 
import android.hardware.camera.common@1.0::types;
 
/**
 *
 * Callback methods for the HAL to call into the framework.
 *
 * These methods are used to return metadata and image buffers for a completed
 * or failed captures, and to notify the framework of asynchronous events such
 * as errors.
 *
 * The framework must not call back into the HAL from within these callbacks,
 * and these calls must not block for extended periods.
 *
 */
interface ICameraDeviceCallback {
 
    /**
     * processCaptureResult:
     *
     * Send results from one or more completed or partially completed captures
     * to the framework.
     * processCaptureResult() may be invoked multiple times by the HAL in
     * response to a single capture request. This allows, for example, the
     * metadata and low-resolution buffers to be returned in one call, and
     * post-processed JPEG buffers in a later call, once it is available. Each
     * call must include the frame number of the request it is returning
     * metadata or buffers for. Only one call to processCaptureResult
     * may be made at a time by the HAL although the calls may come from
     * different threads in the HAL.
     *
     * A component (buffer or metadata) of the complete result may only be
     * included in one process_capture_result call. A buffer for each stream,
     * and the result metadata, must be returned by the HAL for each request in
     * one of the processCaptureResult calls, even in case of errors producing
     * some of the output. A call to processCaptureResult() with neither
     * output buffers or result metadata is not allowed.
     *
     * The order of returning metadata and buffers for a single result does not
     * matter, but buffers for a given stream must be returned in FIFO order. So
     * the buffer for request 5 for stream A must always be returned before the
     * buffer for request 6 for stream A. This also applies to the result
     * metadata; the metadata for request 5 must be returned before the metadata
     * for request 6.
     *
     * However, different streams are independent of each other, so it is
     * acceptable and expected that the buffer for request 5 for stream A may be
     * returned after the buffer for request 6 for stream B is. And it is
     * acceptable that the result metadata for request 6 for stream B is
     * returned before the buffer for request 5 for stream A is. If multiple
     * capture results are included in a single call, camera framework must
     * process results sequentially from lower index to higher index, as if
     * these results were sent to camera framework one by one, from lower index
     * to higher index.
     *
     * The HAL retains ownership of result structure, which only needs to be
     * valid to access during this call.
     *
     * The output buffers do not need to be filled yet; the framework must wait
     * on the stream buffer release sync fence before reading the buffer
     * data. Therefore, this method should be called by the HAL as soon as
     * possible, even if some or all of the output buffers are still in
     * being filled. The HAL must include valid release sync fences into each
     * output_buffers stream buffer entry, or -1 if that stream buffer is
     * already filled.
     *
     * If the result buffer cannot be constructed for a request, the HAL must
     * return an empty metadata buffer, but still provide the output buffers and
     * their sync fences. In addition, notify() must be called with an
     * ERROR_RESULT message.
     *
     * If an output buffer cannot be filled, its status field must be set to
     * STATUS_ERROR. In addition, notify() must be called with a ERROR_BUFFER
     * message.
     *
     * If the entire capture has failed, then this method still needs to be
     * called to return the output buffers to the framework. All the buffer
     * statuses must be STATUS_ERROR, and the result metadata must be an
     * empty buffer. In addition, notify() must be called with a ERROR_REQUEST
     * message. In this case, individual ERROR_RESULT/ERROR_BUFFER messages
     * must not be sent.
     *
     * Performance requirements:
     *
     * This is a non-blocking call. The framework must handle each CaptureResult
     * within 5ms.
     *
     * The pipeline latency (see S7 for definition) should be less than or equal to
     * 4 frame intervals, and must be less than or equal to 8 frame intervals.
     *
     */
    processCaptureResult(vec<CaptureResult> results);
 
    /**
     * notify:
     *
     * Asynchronous notification callback from the HAL, fired for various
     * reasons. Only for information independent of frame capture, or that
     * require specific timing. Multiple messages may be sent in one call; a
     * message with a higher index must be considered to have occurred after a
     * message with a lower index.
     *
     * Multiple threads may call notify() simultaneously.
     *
     * Buffers delivered to the framework must not be dispatched to the
     * application layer until a start of exposure timestamp (or input image's
     * start of exposure timestamp for a reprocess request) has been received
     * via a SHUTTER notify() call. It is highly recommended to dispatch this
     * call as early as possible.
     *
     * ------------------------------------------------------------------------
     * Performance requirements:
     *
     * This is a non-blocking call. The framework must handle each message in 5ms.
     */
    notify(vec<NotifyMsg> msgs);
 
};

     看到这些大段大段的注释，就能明白，这些接口肯定都是非常重要的，详细的注释也是很好的习惯，看Android的源码感觉确实很舒服，难怪人家的代码能成为标准，不管是命名，格式，注释，分包，分类，所有的地方都让人感觉舒服！！

     好了，进入到我们本节的主题吧，ICameraDeviceCallback是HIDL定义的回调接口，processCaptureResult方法就是从HAL层回调到CameraServer的接口，CameraServer这一侧的回调类就是Camera3Device，因为在openCamera时，构造出来的Camera3Device进行初始化，Camera3Device类的initialize方法中与HAL进行连接，获取session时，将自己作为callback回调类传递到了HAL，所以后续HAL就会回调到Camera3Device类的processCaptureResult方法当中。

     我们再来回顾一下Camera3Device类的initialize方法，Camera3Device文件的目录路径为frameworks\av\services\camera\libcameraservice\device3\Camera3Device.cpp，initialize方法的源码如下：

status_t Camera3Device::initialize(sp<CameraProviderManager> manager) {
    ATRACE_CALL();
    Mutex::Autolock il(mInterfaceLock);
    Mutex::Autolock l(mLock);
 
    ALOGV("%s: Initializing HIDL device for camera %s", __FUNCTION__, mId.string());
    if (mStatus != STATUS_UNINITIALIZED) {
        CLOGE("Already initialized!");
        return INVALID_OPERATION;
    }
    if (manager == nullptr) return INVALID_OPERATION;
 
    sp<ICameraDeviceSession> session;
    ATRACE_BEGIN("CameraHal::openSession");
    status_t res = manager->openSession(mId.string(), this,
            /*out*/ &session);
    ATRACE_END();
    if (res != OK) {
        SET_ERR_L("Could not open camera session: %s (%d)", strerror(-res), res);
        return res;
    }
 
    res = manager->getCameraCharacteristics(mId.string(), &mDeviceInfo);
    if (res != OK) {
        SET_ERR_L("Could not retrive camera characteristics: %s (%d)", strerror(-res), res);
        session->close();
        return res;
    }
 
    std::shared_ptr<RequestMetadataQueue> queue;
    auto requestQueueRet = session->getCaptureRequestMetadataQueue(
        [&queue](const auto& descriptor) {
            queue = std::make_shared<RequestMetadataQueue>(descriptor);
            if (!queue->isValid() || queue->availableToWrite() <= 0) {
                ALOGE("HAL returns empty request metadata fmq, not use it");
                queue = nullptr;
                // don't use the queue onwards.
            }
        });
    if (!requestQueueRet.isOk()) {
        ALOGE("Transaction error when getting request metadata fmq: %s, not use it",
                requestQueueRet.description().c_str());
        return DEAD_OBJECT;
    }
    auto resultQueueRet = session->getCaptureResultMetadataQueue(
        [&queue = mResultMetadataQueue](const auto& descriptor) {
            queue = std::make_unique<ResultMetadataQueue>(descriptor);
            if (!queue->isValid() ||  queue->availableToWrite() <= 0) {
                ALOGE("HAL returns empty result metadata fmq, not use it");
                queue = nullptr;
                // Don't use the queue onwards.
            }
        });
    if (!resultQueueRet.isOk()) {
        ALOGE("Transaction error when getting result metadata queue from camera session: %s",
                resultQueueRet.description().c_str());
        return DEAD_OBJECT;
    }
 
    mInterface = std::make_unique<HalInterface>(session, queue);
    std::string providerType;
    mVendorTagId = manager->getProviderTagIdLocked(mId.string());
 
    return initializeCommonLocked();
}

     该方法中就是调用manager->openSession(mId.string(), this, /*out*/ &session)来打开session的，manager是方法入参sp<CameraProviderManager>，调用openSession方法的第二个参数就是回调接口，传值为this，第三个是输出参数，当session在HAL进程中创建成功，则会通过回调赋值给这个输出参数。进一步来看一下CameraProviderManager类的openSession方法的实现，源码如下：

status_t CameraProviderManager::openSession(const std::string &id,
        const sp<hardware::camera::device::V3_2::ICameraDeviceCallback>& callback,
        /*out*/
        sp<hardware::camera::device::V3_2::ICameraDeviceSession> *session) {
 
    std::lock_guard<std::mutex> lock(mInterfaceMutex);
 
    auto deviceInfo = findDeviceInfoLocked(id,
            /*minVersion*/ {3,0}, /*maxVersion*/ {4,0});
    if (deviceInfo == nullptr) return NAME_NOT_FOUND;
 
    auto *deviceInfo3 = static_cast<ProviderInfo::DeviceInfo3*>(deviceInfo);
 
    Status status;
    hardware::Return<void> ret;
    ret = deviceInfo3->mInterface->open(callback, [&status, &session]
            (Status s, const sp<device::V3_2::ICameraDeviceSession>& cameraSession) {
                status = s;
                if (status == Status::OK) {
                    *session = cameraSession;
                }
            });
    if (!ret.isOk()) {
        ALOGE("%s: Transaction error opening a session for camera device %s: %s",
                __FUNCTION__, id.c_str(), ret.description().c_str());
        return DEAD_OBJECT;
    }
    return mapToStatusT(status);
}

     这里的deviceInfo3->mInterface->open就会通过HIDL进入到CameraHalServer进程当中了。

     好，回过头来看Camera3Device类的processCaptureResult方法，源码如下：

// Only one processCaptureResult should be called at a time, so
// the locks won't block. The locks are present here simply to enforce this.
hardware::Return<void> Camera3Device::processCaptureResult(
        const hardware::hidl_vec<
                hardware::camera::device::V3_2::CaptureResult>& results) {
 
    if (mProcessCaptureResultLock.tryLock() != OK) {
        // This should never happen; it indicates a wrong client implementation
        // that doesn't follow the contract. But, we can be tolerant here.
        ALOGE("%s: callback overlapped! waiting 1s...",
                __FUNCTION__);
        if (mProcessCaptureResultLock.timedLock(1000000000 /* 1s */) != OK) {
            ALOGE("%s: cannot acquire lock in 1s, dropping results",
                    __FUNCTION__);
            // really don't know what to do, so bail out.
            return hardware::Void();
        }
    }
    for (const auto& result : results) {
        processOneCaptureResultLocked(result);
    }
    mProcessCaptureResultLock.unlock();
    return hardware::Void();
}