devbox
很楠不爱3
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

RocketMQ源码分析（二）Producer_rocketmq the message body size over max value

作者：很楠不爱3 | 2024-03-17 15:54:53

赞

踩

rocketmq the message body size over max value

1.3 Producer

消息生产者的代码都在client模块中，相对于RocketMQ来讲，消息生产者就是客户端，也是消息的提供者。
在这里插入图片描述

1.3.1 方法和属性

1）主要方法介绍
在这里插入图片描述

    1. //创建主题
    2. void createTopic(final String key, final String newTopic, final int queueNum) throws MQClientException;
    • 1
    • 2
    1. //根据时间戳从队列中查找消息偏移量
    2. long searchOffset(final MessageQueue mq, final long timestamp)
    • 1
    • 2
    1. //查找消息队列中最大的偏移量
    2. long maxOffset(final MessageQueue mq) throws MQClientException;
    • 1
    • 2
    1. //查找消息队列中最小的偏移量
    2. long minOffset(final MessageQueue mq)
    • 1
    • 2
    1. //根据偏移量查找消息
    2. MessageExt viewMessage(final String offsetMsgId) throws RemotingException, MQBrokerException,
    3.         InterruptedException, MQClientException;
    • 1
    • 2
    • 3
    1. //根据条件查找消息
    2. QueryResult queryMessage(final String topic, final String key, final int maxNum, final long begin,
    3.         final long end) throws MQClientException, InterruptedException;
    • 1
    • 2
    • 3
    1. //根据消息ID和主题查找消息
    2. MessageExt viewMessage(String topic,String msgId) throws RemotingException, MQBrokerException, InterruptedException, MQClientException;
    • 1
    • 2

在这里插入图片描述

    1. //启动
    2. void start() throws MQClientException;
    • 1
    • 2
    1. //关闭
    2. void shutdown();
    • 1
    • 2
    1. //查找该主题下所有消息
    2. List<MessageQueue> fetchPublishMessageQueues(final String topic) throws MQClientException;
    • 1
    • 2
    1. //同步发送消息
    2. SendResult send(final Message msg) throws MQClientException, RemotingException, MQBrokerException,
    3.         InterruptedException;
    • 1
    • 2
    • 3
    1. //同步超时发送消息
    2. SendResult send(final Message msg, final long timeout) throws MQClientException,
    3.         RemotingException, MQBrokerException, InterruptedException;
    • 1
    • 2
    • 3
    1. //异步发送消息
    2. void send(final Message msg, final SendCallback sendCallback) throws MQClientException,
    3.         RemotingException, InterruptedException;
    • 1
    • 2
    • 3
    1. //异步超时发送消息
    2. void send(final Message msg, final SendCallback sendCallback, final long timeout)
    3.     throws MQClientException, RemotingException, InterruptedException;
    • 1
    • 2
    • 3
    1. //发送单向消息
    2. void sendOneway(final Message msg) throws MQClientException, RemotingException,
    3.     InterruptedException;
    • 1
    • 2
    • 3
    1. //选择指定队列同步发送消息
    2. SendResult send(final Message msg, final MessageQueue mq) throws MQClientException,
    3.     RemotingException, MQBrokerException, InterruptedException;
    • 1
    • 2
    • 3
    1. //选择指定队列异步发送消息
    2. void send(final Message msg, final MessageQueue mq, final SendCallback sendCallback)
    3.     throws MQClientException, RemotingException, InterruptedException;
    • 1
    • 2
    • 3
    1. //选择指定队列单项发送消息
    2. void sendOneway(final Message msg, final MessageQueue mq) throws MQClientException,
    3.     RemotingException, InterruptedException;
    • 1
    • 2
    • 3
    1. //批量发送消息
    2. SendResult send(final Collection<Message> msgs) throws MQClientException, RemotingException, MQBrokerException,InterruptedException;
    • 1
    • 2

2）属性介绍
在这里插入图片描述

  1. producerGroup：生产者所属组
  2. createTopicKey：默认Topic
  3. defaultTopicQueueNums：默认主题在每一个Broker队列数量
  4. sendMsgTimeout：发送消息默认超时时间，默认3s
  5. compressMsgBodyOverHowmuch：消息体超过该值则启用压缩，默认4k
  6. retryTimesWhenSendFailed：同步方式发送消息重试次数，默认为2，总共执行3次
  7. retryTimesWhenSendAsyncFailed：异步方法发送消息重试次数，默认为2
  8. retryAnotherBrokerWhenNotStoreOK：消息重试时选择另外一个Broker时，是否不等待存储结果就返回，默认为false
  9. maxMessageSize：允许发送的最大消息长度，默认为4M
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9

1.3.2 启动流程

在这里插入图片描述
代码：DefaultMQProducerImpl#start

  1. //检查生产者组是否满足要求
  2. this.checkConfig();
  3. //更改当前instanceName为进程ID
  4. if (!this.defaultMQProducer.getProducerGroup().equals(MixAll.CLIENT_INNER_PRODUCER_GROUP)) {
  5.     this.defaultMQProducer.changeInstanceNameToPID();
  6. }
  7. //获得MQ客户端实例
  8. this.mQClientFactory = MQClientManager.getInstance().getAndCreateMQClientInstance(this.defaultMQProducer, rpcHook);
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

整个JVM中只存在一个MQClientManager实例，维护一个MQClientInstance缓存表

ConcurrentMap<String/* clientId */, MQClientInstance> factoryTable = new ConcurrentHashMap<String,MQClientInstance>();

同一个clientId只会创建一个MQClientInstance。

MQClientInstance封装了RocketMQ网络处理API，是消息生产者和消息消费者与NameServer、Broker打交道的网络通道

代码：MQClientManager#getAndCreateMQClientInstance

  1. public MQClientInstance getAndCreateMQClientInstance(final ClientConfig clientConfig, 
  2.                                                      RPCHook rpcHook) {
  3.     //构建客户端ID
  4.     String clientId = clientConfig.buildMQClientId();
  5.     //根据客户端ID或者客户端实例
  6.     MQClientInstance instance = this.factoryTable.get(clientId);
  7.     //实例如果为空就创建新的实例,并添加到实例表中
  8.     if (null == instance) {
  9.         instance =
  10.             new MQClientInstance(clientConfig.cloneClientConfig(),
  11.                 this.factoryIndexGenerator.getAndIncrement(), clientId, rpcHook);
  12.         MQClientInstance prev = this.factoryTable.putIfAbsent(clientId, instance);
  13.         if (prev != null) {
  14.             instance = prev;
  15.             log.warn("Returned Previous MQClientInstance for clientId:[{}]", clientId);
  16.         } else {
  17.             log.info("Created new MQClientInstance for clientId:[{}]", clientId);
  18.         }
  19.     }
  20.  
  21.     return instance;
  22. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22

代码：DefaultMQProducerImpl#start

  1. //注册当前生产者到到MQClientInstance管理中,方便后续调用网路请求
  2. boolean registerOK = mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this);
  3. if (!registerOK) {
  4.     this.serviceState = ServiceState.CREATE_JUST;
  5.     throw new MQClientException("The producer group[" + this.defaultMQProducer.getProducerGroup()
  6.         + "] has been created before, specify another name please." + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),
  7.         null);
  8. }
  9. //启动生产者
  10. if (startFactory) {
  11.     mQClientFactory.start();
  12. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12

1.3.3 消息发送

在这里插入图片描述
代码：DefaultMQProducerImpl#send(Message msg)

  1. //发送消息
  2. public SendResult send(Message msg) {
  3.     return send(msg, this.defaultMQProducer.getSendMsgTimeout());
  4. }
  • 1
  • 2
  • 3
  • 4

代码：DefaultMQProducerImpl#send(Message msg,long timeout)

  1. //发送消息,默认超时时间为3s
  2. public SendResult send(Message msg,long timeout){
  3.     return this.sendDefaultImpl(msg, CommunicationMode.SYNC, null, timeout);
  4. }
  • 1
  • 2
  • 3
  • 4

代码：DefaultMQProducerImpl#sendDefaultImpl

  1. //校验消息
  2. Validators.checkMessage(msg, this.defaultMQProducer);
  • 1
  • 2

1）验证消息

代码：Validators#checkMessage

  1. public static void checkMessage(Message msg, DefaultMQProducer defaultMQProducer)
  2.     throws MQClientException {
  3.     //判断是否为空
  4.     if (null == msg) {
  5.         throw new MQClientException(ResponseCode.MESSAGE_ILLEGAL, "the message is null");
  6.     }
  7.     // 校验主题
  8.     Validators.checkTopic(msg.getTopic());
  9. 		
  10.     // 校验消息体
  11.     if (null == msg.getBody()) {
  12.         throw new MQClientException(ResponseCode.MESSAGE_ILLEGAL, "the message body is null");
  13.     }
  14.  
  15.     if (0 == msg.getBody().length) {
  16.         throw new MQClientException(ResponseCode.MESSAGE_ILLEGAL, "the message body length is zero");
  17.     }
  18.  
  19.     if (msg.getBody().length > defaultMQProducer.getMaxMessageSize()) {
  20.         throw new MQClientException(ResponseCode.MESSAGE_ILLEGAL,
  21.             "the message body size over max value, MAX: " + defaultMQProducer.getMaxMessageSize());
  22.     }
  23. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23

2）查找路由

代码：DefaultMQProducerImpl#tryToFindTopicPublishInfo

  1. private TopicPublishInfo tryToFindTopicPublishInfo(final String topic) {
  2.     //从缓存中获得主题的路由信息
  3.     TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic);
  4.     //路由信息为空,则从NameServer获取路由
  5.     if (null == topicPublishInfo || !topicPublishInfo.ok()) {
  6.         this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo());
  7.         this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic);
  8.         topicPublishInfo = this.topicPublishInfoTable.get(topic);
  9.     }
  10.  
  11.     if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) {
  12.         return topicPublishInfo;
  13.     } else {
  14.         //如果未找到当前主题的路由信息,则用默认主题继续查找
  15.         this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic, true, this.defaultMQProducer);
  16.         topicPublishInfo = this.topicPublishInfoTable.get(topic);
  17.         return topicPublishInfo;
  18.     }
  19. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19

在这里插入图片描述
代码：TopicPublishInfo

  1. public class TopicPublishInfo {
  2.     private boolean orderTopic = false;	//是否是顺序消息
  3.     private boolean haveTopicRouterInfo = false; 
  4.     private List<MessageQueue> messageQueueList = new ArrayList<MessageQueue>();	//该主题消息队列
  5.     private volatile ThreadLocalIndex sendWhichQueue = new ThreadLocalIndex();//每选择一次消息队列,该值+1
  6.     private TopicRouteData topicRouteData;//关联Topic路由元信息
  7. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

代码：MQClientInstance#updateTopicRouteInfoFromNameServer

  1. TopicRouteData topicRouteData;
  2. //使用默认主题从NameServer获取路由信息
  3. if (isDefault && defaultMQProducer != null) {
  4.     topicRouteData = this.mQClientAPIImpl.getDefaultTopicRouteInfoFromNameServer(defaultMQProducer.getCreateTopicKey(),
  5.         1000 * 3);
  6.     if (topicRouteData != null) {
  7.         for (QueueData data : topicRouteData.getQueueDatas()) {
  8.             int queueNums = Math.min(defaultMQProducer.getDefaultTopicQueueNums(), data.getReadQueueNums());
  9.             data.setReadQueueNums(queueNums);
  10.             data.setWriteQueueNums(queueNums);
  11.         }
  12.     }
  13. } else {
  14.     //使用指定主题从NameServer获取路由信息
  15.     topicRouteData = this.mQClientAPIImpl.getTopicRouteInfoFromNameServer(topic, 1000 * 3);
  16. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16

代码：MQClientInstance#updateTopicRouteInfoFromNameServer

  1. //判断路由是否需要更改
  2. TopicRouteData old = this.topicRouteTable.get(topic);
  3. boolean changed = topicRouteDataIsChange(old, topicRouteData);
  4. if (!changed) {
  5.     changed = this.isNeedUpdateTopicRouteInfo(topic);
  6. } else {
  7.     log.info("the topic[{}] route info changed, old[{}] ,new[{}]", topic, old, topicRouteData);
  8. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

代码：MQClientInstance#updateTopicRouteInfoFromNameServer

  1. if (changed) {
  2.     //将topicRouteData转换为发布队列
  3.     TopicPublishInfo publishInfo = topicRouteData2TopicPublishInfo(topic, topicRouteData);
  4.     publishInfo.setHaveTopicRouterInfo(true);
  5.     //遍历生产
  6.     Iterator<Entry<String, MQProducerInner>> it = this.producerTable.entrySet().iterator();
  7.     while (it.hasNext()) {
  8.         Entry<String, MQProducerInner> entry = it.next();
  9.         MQProducerInner impl = entry.getValue();
  10.         if (impl != null) {
  11.             //生产者不为空时,更新publishInfo信息
  12.             impl.updateTopicPublishInfo(topic, publishInfo);
  13.         }
  14.     }
  15. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15

代码：MQClientInstance#topicRouteData2TopicPublishInfo

  1. public static TopicPublishInfo topicRouteData2TopicPublishInfo(final String topic, final TopicRouteData route) {
  2.     	//创建TopicPublishInfo对象
  3.         TopicPublishInfo info = new TopicPublishInfo();
  4.     	//关联topicRoute
  5.         info.setTopicRouteData(route);
  6.     	//顺序消息,更新TopicPublishInfo
  7.         if (route.getOrderTopicConf() != null && route.getOrderTopicConf().length() > 0) {
  8.             String[] brokers = route.getOrderTopicConf().split(";");
  9.             for (String broker : brokers) {
  10.                 String[] item = broker.split(":");
  11.                 int nums = Integer.parseInt(item[1]);
  12.                 for (int i = 0; i < nums; i++) {
  13.                     MessageQueue mq = new MessageQueue(topic, item[0], i);
  14.                     info.getMessageQueueList().add(mq);
  15.                 }
  16.             }
  17.  
  18.             info.setOrderTopic(true);
  19.         } else {
  20.             //非顺序消息更新TopicPublishInfo
  21.             List<QueueData> qds = route.getQueueDatas();
  22.             Collections.sort(qds);
  23.             //遍历topic队列信息
  24.             for (QueueData qd : qds) {
  25.                 //是否是写队列
  26.                 if (PermName.isWriteable(qd.getPerm())) {
  27.                     BrokerData brokerData = null;
  28.                     //遍历写队列Broker
  29.                     for (BrokerData bd : route.getBrokerDatas()) {
  30.                         //根据名称获得读队列对应的Broker
  31.                         if (bd.getBrokerName().equals(qd.getBrokerName())) {
  32.                         brokerData = bd;
  33.                         break;
  34.                     }
  35.                 }
  36.  
  37.                 if (null == brokerData) {
  38.                     continue;
  39.                 }
  40.  
  41.                 if (!brokerData.getBrokerAddrs().containsKey(MixAll.MASTER_ID)) {
  42.                     continue;
  43.                 }
  44. 				//封装TopicPublishInfo写队列
  45.                 for (int i = 0; i < qd.getWriteQueueNums(); i++) {
  46.                     MessageQueue mq = new MessageQueue(topic, qd.getBrokerName(), i);
  47.                     info.getMessageQueueList().add(mq);
  48.                 }
  49.             }
  50.         }
  51.  
  52.         info.setOrderTopic(false);
  53.     }
  54. 	//返回TopicPublishInfo对象
  55.     return info;
  56. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56

3）选择队列

  • 默认不启用Broker故障延迟机制

代码：TopicPublishInfo#selectOneMessageQueue(lastBrokerName)

  1. public MessageQueue selectOneMessageQueue(final String lastBrokerName) {
  2.     //第一次选择队列
  3.     if (lastBrokerName == null) {
  4.         return selectOneMessageQueue();
  5.     } else {
  6.         //sendWhichQueue
  7.         int index = this.sendWhichQueue.getAndIncrement();
  8.         //遍历消息队列集合
  9.         for (int i = 0; i < this.messageQueueList.size(); i++) {
  10.             //sendWhichQueue自增后取模
  11.             int pos = Math.abs(index++) % this.messageQueueList.size();
  12.             if (pos < 0)
  13.                 pos = 0;
  14.             //规避上次Broker队列
  15.             MessageQueue mq = this.messageQueueList.get(pos);
  16.             if (!mq.getBrokerName().equals(lastBrokerName)) {
  17.                 return mq;
  18.             }
  19.         }
  20.         //如果以上情况都不满足,返回sendWhichQueue取模后的队列
  21.         return selectOneMessageQueue();
  22.     }
  23. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23

代码：TopicPublishInfo#selectOneMessageQueue()

  1. //第一次选择队列
  2. public MessageQueue selectOneMessageQueue() {
  3.     //sendWhichQueue自增
  4.     int index = this.sendWhichQueue.getAndIncrement();
  5.     //对队列大小取模
  6.     int pos = Math.abs(index) % this.messageQueueList.size();
  7.     if (pos < 0)
  8.         pos = 0;
  9.     //返回对应的队列
  10.     return this.messageQueueList.get(pos);
  11. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 启用Broker故障延迟机制
  1. public MessageQueue selectOneMessageQueue(final TopicPublishInfo tpInfo, final String lastBrokerName) {
  2.     //Broker故障延迟机制
  3.     if (this.sendLatencyFaultEnable) {
  4.         try {
  5.             //对sendWhichQueue自增
  6.             int index = tpInfo.getSendWhichQueue().getAndIncrement();
  7.             //对消息队列轮询获取一个队列
  8.             for (int i = 0; i < tpInfo.getMessageQueueList().size(); i++) {
  9.                 int pos = Math.abs(index++) % tpInfo.getMessageQueueList().size();
  10.                 if (pos < 0)
  11.                     pos = 0;
  12.                 MessageQueue mq = tpInfo.getMessageQueueList().get(pos);
  13.                 //验证该队列是否可用
  14.                 if (latencyFaultTolerance.isAvailable(mq.getBrokerName())) {
  15.                     //可用
  16.                     if (null == lastBrokerName || mq.getBrokerName().equals(lastBrokerName))
  17.                         return mq;
  18.                 }
  19.             }
  20. 			//从规避的Broker中选择一个可用的Broker
  21.             final String notBestBroker = latencyFaultTolerance.pickOneAtLeast();
  22.             //获得Broker的写队列集合
  23.             int writeQueueNums = tpInfo.getQueueIdByBroker(notBestBroker);
  24.             if (writeQueueNums > 0) {
  25.                 //获得一个队列,指定broker和队列ID并返回
  26.                 final MessageQueue mq = tpInfo.selectOneMessageQueue();
  27.                 if (notBestBroker != null) {
  28.                     mq.setBrokerName(notBestBroker);
  29.                     mq.setQueueId(tpInfo.getSendWhichQueue().getAndIncrement() % writeQueueNums);
  30.                 }
  31.                 return mq;
  32.             } else {
  33.                 latencyFaultTolerance.remove(notBestBroker);
  34.             }
  35.         } catch (Exception e) {
  36.             log.error("Error occurred when selecting message queue", e);
  37.         }
  38.  
  39.         return tpInfo.selectOneMessageQueue();
  40.     }
  41.  
  42.     return tpInfo.selectOneMessageQueue(lastBrokerName);
  43. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43

在这里插入图片描述

  • 延迟机制接口规范
  1. public interface LatencyFaultTolerance<T> {
  2.     //更新失败条目
  3.     void updateFaultItem(final T name, final long currentLatency, final long notAvailableDuration);
  4. 	//判断Broker是否可用
  5.     boolean isAvailable(final T name);
  6. 	//移除Fault条目
  7.     void remove(final T name);
  8. 	//尝试从规避的Broker中选择一个可用的Broker
  9.     T pickOneAtLeast();
  10. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • FaultItem：失败条目
  1. class FaultItem implements Comparable<FaultItem> {
  2.     //条目唯一键,这里为brokerName
  3.     private final String name;
  4.     //本次消息发送延迟
  5.     private volatile long currentLatency;
  6.     //故障规避开始时间
  7.     private volatile long startTimestamp;
  8. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 消息失败策略
  1. public class MQFaultStrategy {
  2.    //根据currentLatency本地消息发送延迟,从latencyMax尾部向前找到第一个比currentLatency小的索引,如果没有找到,返回0
  3. 	private long[] latencyMax = {50L, 100L, 550L, 1000L, 2000L, 3000L, 15000L};
  4.     //根据这个索引从notAvailableDuration取出对应的时间,在该时长内,Broker设置为不可用
  5. 	private long[] notAvailableDuration = {0L, 0L, 30000L, 60000L, 120000L, 180000L, 600000L};
  6. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6

原理分析

代码：DefaultMQProducerImpl#sendDefaultImpl

  1. sendResult = this.sendKernelImpl(msg, 
  2.                                  mq, 
  3.                                  communicationMode, 
  4.                                  sendCallback, 
  5.                                  topicPublishInfo, 
  6.                                  timeout - costTime);
  7. endTimestamp = System.currentTimeMillis();
  8. this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, false);
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

如果上述发送过程出现异常，则调用DefaultMQProducerImpl#updateFaultItem

  1. public void updateFaultItem(final String brokerName, final long currentLatency, boolean isolation) {
  2.     //参数一：broker名称
  3.     //参数二:本次消息发送延迟时间
  4.     //参数三:是否隔离
  5.     this.mqFaultStrategy.updateFaultItem(brokerName, currentLatency, isolation);
  6. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6

代码：MQFaultStrategy#updateFaultItem

  1. public void updateFaultItem(final String brokerName, final long currentLatency, boolean isolation) {
  2.     if (this.sendLatencyFaultEnable) {
  3.         //计算broker规避的时长
  4.         long duration = computeNotAvailableDuration(isolation ? 30000 : currentLatency);
  5.         //更新该FaultItem规避时长
  6.         this.latencyFaultTolerance.updateFaultItem(brokerName, currentLatency, duration);
  7.     }
  8. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

代码：MQFaultStrategy#computeNotAvailableDuration

  1. private long computeNotAvailableDuration(final long currentLatency) {
  2.     //遍历latencyMax
  3.     for (int i = latencyMax.length - 1; i >= 0; i--) {
  4.         //找到第一个比currentLatency的latencyMax值
  5.         if (currentLatency >= latencyMax[i])
  6.             return this.notAvailableDuration[i];
  7.     }
  8.     //没有找到则返回0
  9.     return 0;
  10. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10

代码：LatencyFaultToleranceImpl#updateFaultItem

  1. public void updateFaultItem(final String name, final long currentLatency, final long notAvailableDuration) {
  2.     //获得原FaultItem
  3.     FaultItem old = this.faultItemTable.get(name);
  4.     //为空新建faultItem对象,设置规避时长和开始时间
  5.     if (null == old) {
  6.         final FaultItem faultItem = new FaultItem(name);
  7.         faultItem.setCurrentLatency(currentLatency);
  8.         faultItem.setStartTimestamp(System.currentTimeMillis() + notAvailableDuration);
  9.  
  10.         old = this.faultItemTable.putIfAbsent(name, faultItem);
  11.         if (old != null) {
  12.             old.setCurrentLatency(currentLatency);
  13.             old.setStartTimestamp(System.currentTimeMillis() + notAvailableDuration);
  14.         }
  15.     } else {
  16.         //更新规避时长和开始时间
  17.         old.setCurrentLatency(currentLatency);
  18.         old.setStartTimestamp(System.currentTimeMillis() + notAvailableDuration);
  19.     }
  20. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20

4）发送消息

消息发送API核心入口DefaultMQProducerImpl#sendKernelImpl

  1. private SendResult sendKernelImpl(
  2.     final Message msg,	//待发送消息
  3.     final MessageQueue mq,	//消息发送队列
  4.     final CommunicationMode communicationMode,		//消息发送内模式
  5.     final SendCallback sendCallback,	pp	//异步消息回调函数
  6.     final TopicPublishInfo topicPublishInfo,	//主题路由信息
  7.     final long timeout	//超时时间
  8.     )
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

代码：DefaultMQProducerImpl#sendKernelImpl

  1. //获得broker网络地址信息
  2. String brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName());
  3. if (null == brokerAddr) {
  4.     //没有找到从NameServer更新broker网络地址信息
  5.     tryToFindTopicPublishInfo(mq.getTopic());
  6.     brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName());
  7. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  1. //为消息分类唯一ID
  2. if (!(msg instanceof MessageBatch)) {
  3.     MessageClientIDSetter.setUniqID(msg);
  4. }
  5.  
  6. boolean topicWithNamespace = false;
  7. if (null != this.mQClientFactory.getClientConfig().getNamespace()) {
  8.     msg.setInstanceId(this.mQClientFactory.getClientConfig().getNamespace());
  9.     topicWithNamespace = true;
  10. }
  11. //消息大小超过4K,启用消息压缩
  12. int sysFlag = 0;
  13. boolean msgBodyCompressed = false;
  14. if (this.tryToCompressMessage(msg)) {
  15.     sysFlag |= MessageSysFlag.COMPRESSED_FLAG;
  16.     msgBodyCompressed = true;
  17. }
  18. //如果是事务消息,设置消息标记MessageSysFlag.TRANSACTION_PREPARED_TYPE
  19. final String tranMsg = msg.getProperty(MessageConst.PROPERTY_TRANSACTION_PREPARED);
  20. if (tranMsg != null && Boolean.parseBoolean(tranMsg)) {
  21.     sysFlag |= MessageSysFlag.TRANSACTION_PREPARED_TYPE;
  22. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  1. //如果注册了消息发送钩子函数,在执行消息发送前的增强逻辑
  2. if (this.hasSendMessageHook()) {
  3.     context = new SendMessageContext();
  4.     context.setProducer(this);
  5.     context.setProducerGroup(this.defaultMQProducer.getProducerGroup());
  6.     context.setCommunicationMode(communicationMode);
  7.     context.setBornHost(this.defaultMQProducer.getClientIP());
  8.     context.setBrokerAddr(brokerAddr);
  9.     context.setMessage(msg);
  10.     context.setMq(mq);
  11.     context.setNamespace(this.defaultMQProducer.getNamespace());
  12.     String isTrans = msg.getProperty(MessageConst.PROPERTY_TRANSACTION_PREPARED);
  13.     if (isTrans != null && isTrans.equals("true")) {
  14.         context.setMsgType(MessageType.Trans_Msg_Half);
  15.     }
  16.  
  17.     if (msg.getProperty("__STARTDELIVERTIME") != null || msg.getProperty(MessageConst.PROPERTY_DELAY_TIME_LEVEL) != null) {
  18.         context.setMsgType(MessageType.Delay_Msg);
  19.     }
  20.     this.executeSendMessageHookBefore(context);
  21. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21

代码：SendMessageHook

  1. public interface SendMessageHook {
  2.     String hookName();
  3.  
  4.     void sendMessageBefore(final SendMessageContext context);
  5.  
  6.     void sendMessageAfter(final SendMessageContext context);
  7. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

代码：DefaultMQProducerImpl#sendKernelImpl

  1. //构建消息发送请求包
  2. SendMessageRequestHeader requestHeader = new SendMessageRequestHeader();
  3. //生产者组
  4. requestHeader.setProducerGroup(this.defaultMQProducer.getProducerGroup());
  5. //主题
  6. requestHeader.setTopic(msg.getTopic());
  7. //默认创建主题Key
  8. requestHeader.setDefaultTopic(this.defaultMQProducer.getCreateTopicKey());
  9. //该主题在单个Broker默认队列树
  10. requestHeader.setDefaultTopicQueueNums(this.defaultMQProducer.getDefaultTopicQueueNums());
  11. //队列ID
  12. requestHeader.setQueueId(mq.getQueueId());
  13. //消息系统标记
  14. requestHeader.setSysFlag(sysFlag);
  15. //消息发送时间
  16. requestHeader.setBornTimestamp(System.currentTimeMillis());
  17. //消息标记
  18. requestHeader.setFlag(msg.getFlag());
  19. //消息扩展信息
  20. requestHeader.setProperties(MessageDecoder.messageProperties2String(msg.getProperties()));
  21. //消息重试次数
  22. requestHeader.setReconsumeTimes(0);
  23. requestHeader.setUnitMode(this.isUnitMode());
  24. //是否是批量消息等
  25. requestHeader.setBatch(msg instanceof MessageBatch);
  26. if (requestHeader.getTopic().startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
  27.     String reconsumeTimes = MessageAccessor.getReconsumeTime(msg);
  28.     if (reconsumeTimes != null) {
  29.         requestHeader.setReconsumeTimes(Integer.valueOf(reconsumeTimes));
  30.         MessageAccessor.clearProperty(msg, MessageConst.PROPERTY_RECONSUME_TIME);
  31.     }
  32.  
  33.     String maxReconsumeTimes = MessageAccessor.getMaxReconsumeTimes(msg);
  34.     if (maxReconsumeTimes != null) {
  35.         requestHeader.setMaxReconsumeTimes(Integer.valueOf(maxReconsumeTimes));
  36.         MessageAccessor.clearProperty(msg, MessageConst.PROPERTY_MAX_RECONSUME_TIMES);
  37.     }
  38. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  1. case ASYNC:		//异步发送
  2.     Message tmpMessage = msg;
  3.     boolean messageCloned = false;
  4.     if (msgBodyCompressed) {
  5.         //If msg body was compressed, msgbody should be reset using prevBody.
  6.         //Clone new message using commpressed message body and recover origin massage.
  7.         //Fix bug:https://github.com/apache/rocketmq-externals/issues/66
  8.         tmpMessage = MessageAccessor.cloneMessage(msg);
  9.         messageCloned = true;
  10.         msg.setBody(prevBody);
  11.     }
  12.  
  13.     if (topicWithNamespace) {
  14.         if (!messageCloned) {
  15.             tmpMessage = MessageAccessor.cloneMessage(msg);
  16.             messageCloned = true;
  17.         }
  18.         msg.setTopic(NamespaceUtil.withoutNamespace(msg.getTopic(), 
  19.                                                     this.defaultMQProducer.getNamespace()));
  20.     }
  21.  
  22. 		long costTimeAsync = System.currentTimeMillis() - beginStartTime;
  23. 		if (timeout < costTimeAsync) {
  24. 		    throw new RemotingTooMuchRequestException("sendKernelImpl call timeout");
  25. 		}
  26. 		sendResult = this.mQClientFactory.getMQClientAPIImpl().sendMessage(
  27.         			brokerAddr,
  28.         			mq.getBrokerName(),
  29.         			tmpMessage,
  30.         			requestHeader,
  31.         			timeout - costTimeAsync,
  32.         			communicationMode,
  33.         			sendCallback,
  34.         			topicPublishInfo,
  35.         			this.mQClientFactory,
  36.         			this.defaultMQProducer.getRetryTimesWhenSendAsyncFailed(),
  37.         			context,
  38.         			this);
  39.     	break;
  40. case ONEWAY:
  41. case SYNC:		//同步发送
  42.     long costTimeSync = System.currentTimeMillis() - beginStartTime;
  43.         if (timeout < costTimeSync) {
  44.             throw new RemotingTooMuchRequestException("sendKernelImpl call timeout");
  45.         }
  46.         sendResult = this.mQClientFactory.getMQClientAPIImpl().sendMessage(
  47.             brokerAddr,
  48.             mq.getBrokerName(),
  49.             msg,
  50.             requestHeader,
  51.             timeout - costTimeSync,
  52.             communicationMode,
  53.             context,
  54.             this);
  55.         break;
  56.     default:
  57.         assert false;
  58.         break;
  59. }
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  1. //如果注册了钩子函数,则发送完毕后执行钩子函数
  2. if (this.hasSendMessageHook()) {
  3.     context.setSendResult(sendResult);
  4.     this.executeSendMessageHookAfter(context);
  5. }
  • 1
  • 2
  • 3
  • 4
  • 5

1.3.4 批量消息发送

在这里插入图片描述
批量消息发送是将同一个主题的多条消息一起打包发送到消息服务端，减少网络调用次数，提高网络传输效率。当然，并不是在同一批次中发送的消息数量越多越好，其判断依据是单条消息的长度，如果单条消息内容比较长，则打包多条消息发送会影响其他线程发送消息的响应时间，并且单批次消息总长度不能超过DefaultMQProducer#maxMessageSize。

批量消息发送要解决的问题是如何将这些消息编码以便服务端能够正确解码出每条消息的消息内容。

代码：DefaultMQProducer#send

  1. public SendResult send(Collection<Message> msgs) 
  2.     throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
  3.     //压缩消息集合成一条消息,然后发送出去
  4.     return this.defaultMQProducerImpl.send(batch(msgs));
  5. }
  • 1
  • 2
  • 3
  • 4
  • 5

代码：DefaultMQProducer#batch

  1. private MessageBatch batch(Collection<Message> msgs) throws MQClientException {
  2.     MessageBatch msgBatch;
  3.     try {
  4.         //将集合消息封装到MessageBatch
  5.         msgBatch = MessageBatch.generateFromList(msgs);
  6.         //遍历消息集合,检查消息合法性,设置消息ID,设置Topic
  7.         for (Message message : msgBatch) {
  8.             Validators.checkMessage(message, this);
  9.             MessageClientIDSetter.setUniqID(message);
  10.             message.setTopic(withNamespace(message.getTopic()));
  11.         }
  12.         //压缩消息,设置消息body
  13.         msgBatch.setBody(msgBatch.encode());
  14.     } catch (Exception e) {
  15.         throw new MQClientException("Failed to initiate the MessageBatch", e);
  16.     }
  17.     //设置msgBatch的topic
  18.     msgBatch.setTopic(withNamespace(msgBatch.getTopic()));
  19.     return msgBatch;
  20. }
声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/很楠不爱3/article/detail/256899
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号