#BHEADER********************************************************************** # Copyright (c) 2008, Lawrence Livermore National Security, LLC. # Produced at the Lawrence Livermore National Laboratory. # This file is part of HYPRE. See file COPYRIGHT for details. # # HYPRE is free software; you can redistribute it and/or modify it under the # terms of the GNU Lesser General Public License (as published by the Free # Software Foundation) version 2.1 dated February 1999. # # $Revision: 2.4 $ #EHEADER********************************************************************** README for the hypre_DataExchangeList() function ************** * Purpose: * ************** This function allows a processor (i) to "contact" a list of processors with a message containing a variable amount of data. The processors in the list do not know that they will be contacted by processor i. These processors must then send a "response" message back to processor i, and the response message can also contain any amount of data. Essentially, the user gives the function a list of processors and corresponding lists of data (of any type) to send to each processor. The function returns lists of data (the "responses") from each of the contacted processors. The user must write a function that dictates how to create a response message based on a contact message. *************** * Function: * *************** int hypre_DataExchangeList(int num_contacts, int *contact_proc_list, void *contact_send_buf, int *contact_send_buf_starts, int contact_obj_size, int response_obj_size, hypre_DataExchangeResponse *response_obj, int max_response_size, int rnum, MPI_Comm comm, void **p_response_recv_buf, int **p_response_recv_buf_starts) ***************** * Parameters: * ***************** num_contacts = the number of processors to contact contact_proc_list = list of processors to contact contact_send_buf = array of data to send - data may be any type (ints, doubles, structures, etc.). contact_send_buf_starts = index into contact_send_buf corresponding to contact_proc_list. For example, processor (contact_proc_list[j]) is sent the buffer contact_send_buf[contact_send_buf_starts[j]], and this buffer is of size (contact_send_buf_starts[j+1] - contact_send_buf_starts[j]). contact_obj_size = sizeof() one data item (int, double, structure, etc.) in contact_send_buf. THIS NEEDS TO BE THE SAME NUMBER FOR ALL PROCESSORS. response_obj_size = sizeof() one data item in response_recv_buf(int, double, structure, etc.) THIS NEEDS TO BE THE SAME NUMBER FOR ALL PROCESSORS. response_obj = this will give us the function we need to fill the response as well as any data we might need to accomplish that (more details below.) max_response_size - this number indicates the size of the array of data that the contacting processor expects back in a single response. This should be a reasonable estimate - not an absolute upper bound! If a contacted processor needs to respond with more than max_response_size data, then a second communication is used - this is transparent to the user and occurs inside hypre_DataExchangeList(). Choosing a max_response_size that is way too large will result in the passing of overly large messages. THIS NEEDS TO BE THE SAME NUMBER FOR ALL PROCESSORS. rnum = two consecutive calls to this function should have different rnums. Recommend alternating rnum = 1 and rnum=2 - these flags will be even (so odd numbered tags could be used in calling code in between). THIS NEEDS TO BE THE SAME NUMBER FOR ALL PROCESSORS. comm = MPI_COMM p_response_recv_buf = (OUTPUT) where to receive the responses - memory will be allocated in this function (do not preallocate) p_response_recv_buf_starts = (OUTPUT) index of p_response_buf corresponding to contact_buf_list - memory will be allocated in this function ********************** * Response Object: * ********************** This object has a function pointer to assign before including it in a call to hypre_DataExchangeList(). In addition, one or more of the void pointers may be set. Each item in the structure is explained below. typedef struct { int (*fill_response)(void* recv_buf, int contact_size, int contact_proc, void* response_obj, MPI_Comm comm, void** response_buf, int* response_message_size, ); int send_response_overhead; int send_response_storage; void *data1; void *data2; } hypre_DataExchangeResponse; int (*fill_response)(void* recv_buf, int contact_size, int contact_proc, void* response_obj, MPI_Comm comm, void** response_buf, int* response_message_size, ); The user writes a function to assign to this pointer. This function is called by hypre_DataExchangeList() upon receipt of a contact message and must determine a response. The data from the contact message is stored in recv_buf, where contact_size indicates the number of objects and contact_proc indicates the processor that made the contact. The user will know the data type and can manipulate recv_buf accordingly. Likewise, the user can populate response_buf as desired - this will constitute the response message. The user must indicate the size of the response in response_message_size. Note that response_message_size and contact_size are NOT in bytes - but rather the number of objects sent (i.e. contact size = 4 if four integers or structures were sent). Note 1: If the desired response to contact messages is a NULL response (a confirmation), simply assign response_message_size to zero - do NOT set the response_buf to NULL. Note 2: cast the response_obj argument to a hypre_DataExchangeResponse object to access the data fields. send_response_storage - this number indicates the amount of storage (again not in bytes) available for the response_buf objects. The user should make sure they do not need to write a larger amount of data than is available. If more storage is needed then adjust the send_response_storage variable to indicate the new size desired. Then realloc response_buf to size (send_response_storage + send_response_overhead)*(size of response object). send_response_overhead - this is extra overhead needed by hypre_DataExchangeList. Include this if you need to realloc response_buf (see previous entry). This is determined by the hypre_DataExchangeFunction(). data1, data2 - these are void pointers that the user may want to assign to data from the program that calls exchangeDataList() in the event that (1) certain data is needed to formulate an appropriate response or (2) data needs to be saved from the contacts to be manipulated after hypre_DataExchangeList() completes.