Java通用并发调度框架

1.背景

为提高API接口性能,一般需要API接口依赖的对部分其它服务接口调用进行并行化处理。之前逻辑是采ThreadPoolExecutor+CountDownLatch进行并行调度控制,该方案有个较大缺点为很容易造成代码膨胀和不可维护,需要开发一个新的调度框架来提高并行调度的友好程度。

2.解决方案

开发一套并行调度框架,底层逻辑仍然为ThreadPoolExecutor+CountDownLatch,降低并行开发接入难度。

2.1执行单元抽象类TCallable

public class TCallable<T> {

    private TPWorker worker;

    private Callable callable;

    private T result;

    public TCallable(Callable callable) {
        this.callable = callable;
    }


    /**
     * 执行任务
     */
    public void execute() {
        try {
            if (Objects.isNull(callable)) {
                // 抛出业务异常
                throw new BuzException("执行任务前请先添加任务");
            }
            this.result = (T) callable.call();
        } catch (Exception ex) {
            log.error("TRunnable error occurs", ex);
        } finally {
            worker.getCountDownLatch().countDown();
        }
    }

    /**
     * 获取结果
     *
     * @return
     */
    public T getResult() {
        try {
            worker.getCountDownLatch().await(worker.getTimeout(), TimeUnit.MILLISECONDS);
        } catch (InterruptedException ex) {
            log.error("getResult error occurs", ex);
            Thread.currentThread().interrupt();
        }
        return result;
    }
}

2.2 流程控制类TPWorker

public class TPWorker {

    private CountDownLatch countDownLatch;
    private Integer timeout = 2000;

    private List<TCallable> jobs = Lists.newArrayList();

    public TPWorker(Integer timeout) {
        this.timeout = timeout;
    }

    public TPWorker() {
    }


    public TPWorker addJob(TCallable runnable) {
        runnable.setWorker(this);
        this.jobs.add(runnable);
        return this;
    }

    public void executeAll() {
        this.countDownLatch = new CountDownLatch(jobs.size());
        for (TCallable callable : jobs) {
            ConcurrentUtils.THREAD_POOL_EXECUTOR.execute(() -> callable.execute());
        }
        waitAllFinish();
    }

    /**
     * 等待并发匹配结束
     */
    private void waitAllFinish() {
        try {
            boolean waitResult = countDownLatch.await(timeout, TimeUnit.MILLISECONDS);
            if (!waitResult) {
                log.warn("waitAllFinish 等待结果超时! ");
            }
        } catch (InterruptedException ex) {
            log.warn("waitAllFinish-被中断!", ex);
            Thread.currentThread().interrupt();
        }
    }
}

2.3 线程池管理类ConcurrentUtils

public class ConcurrentUtils {
    private static final int THREAD_CORE_NUM = 60;
    private static final int THREAD_MAX_NUM = 300;
    public static final ThreadPoolExecutor THREAD_POOL_EXECUTOR;

    static {
        ThreadFactory processorFactory = new ThreadFactoryBuilder().setNameFormat("TPWorker-%d").build();
        THREAD_POOL_EXECUTOR = new ThreadPoolExecutor(THREAD_CORE_NUM,
                THREAD_MAX_NUM,
                Integer.MAX_VALUE,
                TimeUnit.MILLISECONDS,
                new LinkedBlockingQueue<>(1000),
                processorFactory,
                new ThreadPoolExecutor.CallerRunsPolicy()
        );
    }
}

2.4 代码范例

从示例代码可知,接入成本非常低;接入并行调度框架后,代码结构上更加清晰易懂

TCallable<List<VipInfo>> listVipJob = new TCallable<>(() -> listVipInfo(conditionDos, vipTypeIds, priceQueryReq.getUid()));
TCallable<Map<Integer, Map<String, String>>> getTagJob = new TCallable<>(() -> getVipTagMap(vipTypeIds, queryTags, priceQueryReq.getUid()));
// 执行并行任务
new TPWorker(cloudConfig.getIntProperty(TPWORKER_PROCESS_TIMEOUT, 3000))
        .addJob(listVipJob)
        .addJob(getTagJob)
        .executeAll();

List<VipInfo> vipInfos = listVipJob.getResult();
Map<Integer, Map<String, String>> tagMap = getTagJob.getResult()

3.总结与展望

  • 从代码非常示例可知,接入该并发调度框架的成本非常低,降低了接入并行编程的难度,提高了代码的可读性和可维护性
  • 本调度框架仅可解决无依赖关系的耗时操作并行执行问题,如果需要进行有依赖关系的复杂并行调度,各个调度任务类似图的形式,可以参考Linkedin更加成(复)熟(杂)的并行调度解决方案parseq
相关推荐
©️2020 CSDN 皮肤主题: 编程工作室 设计师:CSDN官方博客 返回首页