HazelcastAPI MapLoader Arayüzü - Harici Bir Kaynaktan Veri Yükler

Giriş

Şu satırı dahil ederiz

import com.hazelcast.map.MapLoader;

MapLoader gerekirse MapLoaderLifecycleSupport arayüzünden kalıtabilir. 

Açıklaması şöyle

MapLoader is never executed from client thread, always from a separate thread pool

Eski Yöntem

Açıklaması şöyle

Here is the MapLoader initialization flow:

1. When getMap() is first called from any member, initialization starts depending on the value of the initial-mode property. If it is set to EAGER, initialization starts on all partitions as soon as the map is created, i.e., all partitions are loaded when getMap is called. A map is loaded when the IMap#getMap method is called in the EAGER mode; otherwise, it is loaded after the first operation, e.g., IMap#size, IMap#get, etc. of the map in the LAZY mode.

2. Hazelcast calls the MapLoader.loadAllKeys() to get all your keys on one of the members.

3. That member distributes keys to all other members in batches.

4. Each member loads values of all its owned keys by calling MapLoader.loadAll(keys).

5. Each member puts its owned entries into the map by calling IMap.putTransient(key,value).

Açıklaması şöyle

Probably in Hazelcast 3.4 we are going to enrich the loading model. I believe there will be more control on loading all keys per partition, because currently 1 machine is going to be 'victim' to load all keys, but in some cases it is better to spread the load. 

Yeni Yöntem

Not : Varsayılan MapStoreConfig.InitialLoadMode değeri LAZY

Açıklaması şöyle. Sanırım her partition kendisine ait değerleri yüklüyor

Each partition is loaded when it is first touched.

loadAll metodu

İmzası şöyle. Eğer key için value yoksa null döndürülemez!

Map<K, V> loadAll(Collection<K> keys);

loadAllKeys metodu

Map ilk yaratıldığında üyelerden sadece bir tanesi tarafından çalıştırılır. Key nesnelerini yükler. Açıklaması şöyle

This method is called when the Hazelcast IMap is first created, and it is used to preload all the keys from the data store into the map. By preloading the keys, you can avoid having to load the keys lazily when they are accessed for the first time, which can improve performance.

Açıklaması şöyle. Yani null, kısmi bir key Iterable veya her şeyi içeren bir Iterable döndürülebilir

The MapLoader.loadAllKeys() method can return all, some, or none of the keys. For example, you can specify a range of keys to be loaded, then rely on read-through to load the remaining keys on demand. Or, you can return a null value so that no data is loaded from the data store.

If the number of keys to load is large, it is more efficient to load them incrementally rather than loading them all at once. To support incremental loading, the MapLoader.loadAllKeys() method returns an Iterable which can be lazily populated with the results of a database query. Hazelcast iterates over the returned data and, while doing so, sends the keys to their respective owner members. The iterator that was returned from the MapLoader.loadAllKeys() method may also implement the Closeable interface, in which case the iterator is closed when the iteration is over. This is intended for releasing resources such as closing a JDBC result set.

Call stack şöyle

- MyMapLoader.loadAllKeys

- MapStoreWrapper.loadAllKeys

- BasicMapStoreContext.loadAllKeys : BasicMapStoreContext Iterable<K> null ise boş collection döner. Bu yüzden metod null döndürebilir

- MapKeyLoader.sendKeysInBatches

- MapKeyLoader.sendKeys

MapKeyLoader kodu şöyle. sendKeysInBatches ve dolayısıyla geri kalan her şey bir başka thread içinde çalıştırılıyor

private Future<?> sendKeys(final MapStoreContext mapStoreContext, 
                           final boolean replaceExistingValues) {
  if (keyLoadFinished.isDone()) {
    keyLoadFinished = new LoadFinishedFuture();

    Future<Boolean> sent = execService.submit(MAP_LOAD_ALL_KEYS_EXECUTOR, () -> {
      sendKeysInBatches(mapStoreContext, replaceExistingValues);
      return false;
    });

    execService.asCompletableFuture(sent).whenCompleteAsync(keyLoadFinished,
                    ConcurrencyUtil.getDefaultAsyncExecutor());
    }

    return keyLoadFinished;
}

Örnek

Şöyle yaparız. Bu örnekte veri tabanındaki her şey bir anda yükleniyor.

public class MyMapLoader implements MapLoader<Integer, String> {
    
  // other methods
    
  @Override
  public Iterable<Integer> loadAllKeys() {
    // load all the keys from the data store and return them as a set
    Set<Integer> keys = new HashSet<>();
    try (Connection conn = dataSource.getConnection();
      Statement stmt = conn.createStatement();
      ResultSet rs = stmt.executeQuery("SELECT id FROM my_table")) {
      while (rs.next()) {
        int id = rs.getInt("id");
        keys.add(id);
      }
    } catch (SQLException e) {
      throw new RuntimeException("Error loading keys", e);
    }
    return keys;
  }
}