ufo22940268
V2EX  ›  问与答

spark 中怎么实现 intersectionByKey

  •  
  •   ufo22940268 · Aug 7, 2017 · 1748 views
    This topic created in 3294 days ago, the information mentioned may be changed or developed.

    问题就是加入我现在有两个 object array, 然后我要取两者公共的 object

    https://forums.databricks.com/questions/1980/filtering-an-rdd-depending-upon-a-list-of-values-i.html

    我在上面链接上找到一个解决方案,但是我觉得这个解决方案是不是效率不高

    1 replies    2017-08-07 11:21:34 +08:00
    ufo22940268
        1
    ufo22940268  
    OP
       Aug 7, 2017
    尴尬, 刚发完自己就研究出一种写法

    https://gist.github.com/ufo22940268/9fb56db23942dae943527175cd05c3c6

    这样子写的话,就算 a 中有相同的 key, 也最终能够做到 intersectionByKey
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   2725 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 32ms · UTC 12:14 · PVG 20:14 · LAX 05:14 · JFK 08:14
    ♥ Do have faith in what you're doing.