$extrastylesheet
libMesh::SparsityPattern::Build Class Reference

#include <sparsity_pattern.h>

Inheritance diagram for libMesh::SparsityPattern::Build:

List of all members.

Public Member Functions

 Build (const MeshBase &mesh_in, const DofMap &dof_map_in, const CouplingMatrix *dof_coupling_in, const bool implicit_neighbor_dofs_in, const bool need_full_sparsity_pattern_in)
 Build (Build &other, Threads::split)
void operator() (const ConstElemRange &range)
void join (const Build &other)
void parallel_sync ()
const Parallel::Communicatorcomm () const
processor_id_type n_processors () const
processor_id_type processor_id () const

Public Attributes

SparsityPattern::Graph sparsity_pattern
SparsityPattern::NonlocalGraph nonlocal_pattern
std::vector< dof_id_typen_nz
std::vector< dof_id_typen_oz

Protected Attributes

const Parallel::Communicator_communicator

Private Attributes

const MeshBasemesh
const DofMapdof_map
const CouplingMatrixdof_coupling
const bool implicit_neighbor_dofs
const bool need_full_sparsity_pattern

Detailed Description

This helper class can be called on multiple threads to compute the sparsity pattern (or graph) of the sparse matrix resulting from the discretization. This pattern may be used directly by a particular sparse matrix format (e.g. LaspackMatrix) or indirectly (e.g. PetscMatrix). In the latter case the number of nonzeros per row of the matrix is needed for efficient preallocation. In this case it suffices to provide estimate (but bounding) values, and in this case the threaded method can take some short-cuts for efficiency.

Definition at line 79 of file sparsity_pattern.h.


Constructor & Destructor Documentation

libMesh::SparsityPattern::Build::Build ( const MeshBase mesh_in,
const DofMap dof_map_in,
const CouplingMatrix dof_coupling_in,
const bool  implicit_neighbor_dofs_in,
const bool  need_full_sparsity_pattern_in 
)

Definition at line 35 of file sparsity_pattern.C.

                                                        :
  ParallelObject(dof_map_in),
  mesh(mesh_in),
  dof_map(dof_map_in),
  dof_coupling(dof_coupling_in),
  implicit_neighbor_dofs(implicit_neighbor_dofs_in),
  need_full_sparsity_pattern(need_full_sparsity_pattern_in),
  sparsity_pattern(),
  nonlocal_pattern(),
  n_nz(),
  n_oz()
{}

Definition at line 54 of file sparsity_pattern.C.

                                        :
  ParallelObject(other),
  mesh(other.mesh),
  dof_map(other.dof_map),
  dof_coupling(other.dof_coupling),
  implicit_neighbor_dofs(other.implicit_neighbor_dofs),
  need_full_sparsity_pattern(other.need_full_sparsity_pattern),
  sparsity_pattern(),
  nonlocal_pattern(),
  n_nz(),
  n_oz()
{}

Member Function Documentation

const Parallel::Communicator& libMesh::ParallelObject::comm ( ) const [inline, inherited]
Returns:
a reference to the Parallel::Communicator object used by this mesh.

Definition at line 86 of file parallel_object.h.

References libMesh::ParallelObject::_communicator.

Referenced by libMesh::__libmesh_petsc_diff_solver_monitor(), libMesh::__libmesh_petsc_diff_solver_residual(), libMesh::__libmesh_petsc_snes_residual(), libMesh::MeshRefinement::_coarsen_elements(), libMesh::ExactSolution::_compute_error(), libMesh::MetisPartitioner::_do_partition(), libMesh::ParmetisPartitioner::_do_repartition(), libMesh::UniformRefinementEstimator::_estimate_error(), libMesh::SlepcEigenSolver< T >::_petsc_shell_matrix_get_diagonal(), libMesh::PetscLinearSolver< T >::_petsc_shell_matrix_get_diagonal(), libMesh::SlepcEigenSolver< T >::_petsc_shell_matrix_mult(), libMesh::PetscLinearSolver< T >::_petsc_shell_matrix_mult(), libMesh::PetscLinearSolver< T >::_petsc_shell_matrix_mult_add(), libMesh::EquationSystems::_read_impl(), libMesh::MeshRefinement::_refine_elements(), libMesh::ImplicitSystem::add_matrix(), libMesh::System::add_vector(), libMesh::UnstructuredMesh::all_second_order(), libMesh::LaplaceMeshSmoother::allgather_graph(), libMesh::FEMSystem::assemble_qoi(), libMesh::MeshCommunication::assign_global_indices(), libMesh::ParmetisPartitioner::assign_partitioning(), libMesh::DofMap::attach_matrix(), libMesh::MeshTools::bounding_box(), libMesh::MeshBase::cache_elem_dims(), libMesh::System::calculate_norm(), libMesh::MeshRefinement::coarsen_elements(), libMesh::Nemesis_IO_Helper::compute_num_global_elem_blocks(), libMesh::Nemesis_IO_Helper::compute_num_global_nodesets(), libMesh::Nemesis_IO_Helper::compute_num_global_sidesets(), libMesh::Problem_Interface::computeF(), libMesh::Problem_Interface::computeJacobian(), libMesh::Problem_Interface::computePreconditioner(), libMesh::MeshTools::correct_node_proc_ids(), libMesh::MeshCommunication::delete_remote_elements(), libMesh::DofMap::distribute_dofs(), DMlibMeshFunction(), DMlibMeshSetSystem_libMesh(), libMesh::MeshRefinement::eliminate_unrefined_patches(), libMesh::WeightedPatchRecoveryErrorEstimator::estimate_error(), libMesh::PatchRecoveryErrorEstimator::estimate_error(), libMesh::JumpErrorEstimator::estimate_error(), libMesh::AdjointRefinementEstimator::estimate_error(), libMesh::ExactErrorEstimator::estimate_error(), libMesh::MeshRefinement::flag_elements_by_elem_fraction(), libMesh::MeshRefinement::flag_elements_by_error_fraction(), libMesh::MeshRefinement::flag_elements_by_nelem_target(), libMesh::CondensedEigenSystem::get_eigenpair(), libMesh::ImplicitSystem::get_linear_solver(), libMesh::LocationMap< T >::init(), libMesh::TimeSolver::init(), libMesh::SystemSubsetBySubdomain::init(), libMesh::EigenSystem::init_data(), libMesh::EigenSystem::init_matrices(), libMesh::ParmetisPartitioner::initialize(), libMesh::MeshTools::libmesh_assert_valid_dof_ids(), libMesh::ParallelMesh::libmesh_assert_valid_parallel_flags(), libMesh::MeshTools::libmesh_assert_valid_procids< Elem >(), libMesh::MeshTools::libmesh_assert_valid_procids< Node >(), libMesh::MeshTools::libmesh_assert_valid_refinement_flags(), libMesh::MeshRefinement::limit_level_mismatch_at_edge(), libMesh::MeshRefinement::limit_level_mismatch_at_node(), libMesh::MeshRefinement::make_coarsening_compatible(), libMesh::MeshCommunication::make_elems_parallel_consistent(), libMesh::MeshRefinement::make_flags_parallel_consistent(), libMesh::MeshCommunication::make_node_ids_parallel_consistent(), libMesh::MeshCommunication::make_node_proc_ids_parallel_consistent(), libMesh::MeshCommunication::make_nodes_parallel_consistent(), libMesh::MeshRefinement::make_refinement_compatible(), libMesh::FEMSystem::mesh_position_set(), libMesh::MeshSerializer::MeshSerializer(), libMesh::ParallelMesh::n_active_elem(), libMesh::MeshTools::n_active_levels(), libMesh::BoundaryInfo::n_boundary_conds(), libMesh::BoundaryInfo::n_edge_conds(), libMesh::CondensedEigenSystem::n_global_non_condensed_dofs(), libMesh::MeshTools::n_levels(), libMesh::BoundaryInfo::n_nodeset_conds(), libMesh::MeshTools::n_p_levels(), libMesh::ParallelMesh::parallel_max_elem_id(), libMesh::ParallelMesh::parallel_max_node_id(), libMesh::ParallelMesh::parallel_n_elem(), libMesh::ParallelMesh::parallel_n_nodes(), libMesh::Partitioner::partition(), libMesh::Partitioner::partition_unpartitioned_elements(), libMesh::petsc_auto_fieldsplit(), libMesh::System::point_gradient(), libMesh::System::point_hessian(), libMesh::System::point_value(), libMesh::MeshBase::prepare_for_use(), libMesh::System::project_vector(), libMesh::Nemesis_IO::read(), libMesh::XdrIO::read(), libMesh::System::read_header(), libMesh::System::read_legacy_data(), libMesh::System::read_SCALAR_dofs(), libMesh::XdrIO::read_serialized_bc_names(), libMesh::XdrIO::read_serialized_bcs(), libMesh::System::read_serialized_blocked_dof_objects(), libMesh::XdrIO::read_serialized_connectivity(), libMesh::XdrIO::read_serialized_nodes(), libMesh::XdrIO::read_serialized_nodesets(), libMesh::XdrIO::read_serialized_subdomain_names(), libMesh::System::read_serialized_vector(), libMesh::MeshBase::recalculate_n_partitions(), libMesh::MeshRefinement::refine_and_coarsen_elements(), libMesh::MeshRefinement::refine_elements(), libMesh::Partitioner::set_node_processor_ids(), libMesh::DofMap::set_nonlocal_dof_objects(), libMesh::LaplaceMeshSmoother::smooth(), libMesh::MeshBase::subdomain_ids(), libMesh::BoundaryInfo::sync(), libMesh::Parallel::sync_element_data_by_parent_id(), libMesh::Parallel::sync_node_data_by_element_id(), libMesh::MeshRefinement::test_level_one(), libMesh::MeshRefinement::test_unflagged(), libMesh::MeshTools::total_weight(), libMesh::NameBasedIO::write(), libMesh::CheckpointIO::write(), libMesh::XdrIO::write(), libMesh::LegacyXdrIO::write_mesh(), libMesh::System::write_SCALAR_dofs(), libMesh::XdrIO::write_serialized_bcs(), libMesh::System::write_serialized_blocked_dof_objects(), libMesh::XdrIO::write_serialized_connectivity(), libMesh::XdrIO::write_serialized_nodes(), libMesh::XdrIO::write_serialized_nodesets(), and libMesh::DivaIO::write_stream().

  { return _communicator; }

Definition at line 2845 of file dof_map.C.

References dof_id, libMesh::libmesh_assert(), std::min(), n_nz, n_oz, nonlocal_pattern, libMesh::processor_id(), libMesh::ParallelObject::processor_id(), and sparsity_pattern.

{
  const processor_id_type proc_id           = mesh.processor_id();
  const dof_id_type       n_global_dofs     = dof_map.n_dofs();
  const dof_id_type       n_dofs_on_proc    = dof_map.n_dofs_on_processor(proc_id);
  const dof_id_type       first_dof_on_proc = dof_map.first_dof(proc_id);
  const dof_id_type       end_dof_on_proc   = dof_map.end_dof(proc_id);

  libmesh_assert_equal_to (sparsity_pattern.size(), other.sparsity_pattern.size());
  libmesh_assert_equal_to (n_nz.size(), sparsity_pattern.size());
  libmesh_assert_equal_to (n_oz.size(), sparsity_pattern.size());

  for (dof_id_type r=0; r<n_dofs_on_proc; r++)
    {
      // increment the number of on and off-processor nonzeros in this row
      // (note this will be an upper bound unless we need the full sparsity pattern)
      if (need_full_sparsity_pattern)
        {
          SparsityPattern::Row       &my_row    = sparsity_pattern[r];
          const SparsityPattern::Row &their_row = other.sparsity_pattern[r];

          // simple copy if I have no dofs
          if (my_row.empty())
            my_row = their_row;

          // otherwise add their DOFs to mine, resort, and re-unique the row
          else if (!their_row.empty()) // do nothing for the trivial case where
            {                          // their row is empty
              my_row.insert (my_row.end(),
                             their_row.begin(),
                             their_row.end());

              // We cannot use SparsityPattern::sort_row() here because it expects
              // the [begin,middle) [middle,end) to be non-overlapping.  This is not
              // necessarily the case here, so use std::sort()
              std::sort (my_row.begin(), my_row.end());

              my_row.erase(std::unique (my_row.begin(), my_row.end()), my_row.end());
            }

          // fix the number of on and off-processor nonzeros in this row
          n_nz[r] = n_oz[r] = 0;

          for (std::size_t j=0; j<my_row.size(); j++)
            if ((my_row[j] < first_dof_on_proc) || (my_row[j] >= end_dof_on_proc))
              n_oz[r]++;
            else
              n_nz[r]++;
        }
      else
        {
          n_nz[r] += other.n_nz[r];
          n_nz[r] = std::min(n_nz[r], n_dofs_on_proc);
          n_oz[r] += other.n_oz[r];
          n_oz[r] =std::min(n_oz[r], static_cast<dof_id_type>(n_global_dofs-n_nz[r]));
        }
    }

  // Move nonlocal row information to ourselves; the other thread
  // won't need it in the map after that.
  NonlocalGraph::const_iterator it = other.nonlocal_pattern.begin();
  for (; it != other.nonlocal_pattern.end(); ++it)
    {
#ifndef NDEBUG
      const dof_id_type dof_id = it->first;

      processor_id_type dbg_proc_id = 0;
      while (dof_id >= dof_map.end_dof(dbg_proc_id))
        dbg_proc_id++;
      libmesh_assert (dbg_proc_id != this->processor_id());
#endif

      const SparsityPattern::Row &their_row = it->second;

      // We should have no empty values in a map
      libmesh_assert (!their_row.empty());

      NonlocalGraph::iterator my_it = nonlocal_pattern.find(it->first);
      if (my_it == nonlocal_pattern.end())
        {
          //          nonlocal_pattern[it->first].swap(their_row);
          nonlocal_pattern[it->first] = their_row;
        }
      else
        {
          SparsityPattern::Row &my_row = my_it->second;

          my_row.insert (my_row.end(),
                         their_row.begin(),
                         their_row.end());

          // We cannot use SparsityPattern::sort_row() here because it expects
          // the [begin,middle) [middle,end) to be non-overlapping.  This is not
          // necessarily the case here, so use std::sort()
          std::sort (my_row.begin(), my_row.end());

          my_row.erase(std::unique (my_row.begin(), my_row.end()), my_row.end());
        }
    }
}
Returns:
the number of processors in the group.

Definition at line 92 of file parallel_object.h.

References libMesh::ParallelObject::_communicator, and libMesh::Parallel::Communicator::size().

Referenced by libMesh::ParmetisPartitioner::_do_repartition(), libMesh::ParallelMesh::add_elem(), libMesh::ParallelMesh::add_node(), libMesh::LaplaceMeshSmoother::allgather_graph(), libMesh::FEMSystem::assembly(), libMesh::ParmetisPartitioner::assign_partitioning(), libMesh::ParallelMesh::assign_unique_ids(), libMesh::AztecLinearSolver< T >::AztecLinearSolver(), libMesh::ParallelMesh::clear(), libMesh::Nemesis_IO_Helper::compute_border_node_ids(), libMesh::Nemesis_IO_Helper::construct_nemesis_filename(), libMesh::UnstructuredMesh::create_pid_mesh(), libMesh::DofMap::distribute_dofs(), libMesh::DofMap::distribute_local_dofs_node_major(), libMesh::DofMap::distribute_local_dofs_var_major(), libMesh::EnsightIO::EnsightIO(), libMesh::MeshBase::get_info(), libMesh::EquationSystems::init(), libMesh::SystemSubsetBySubdomain::init(), libMesh::ParmetisPartitioner::initialize(), libMesh::Nemesis_IO_Helper::initialize(), libMesh::MeshTools::libmesh_assert_valid_dof_ids(), libMesh::MeshTools::libmesh_assert_valid_procids< Elem >(), libMesh::MeshTools::libmesh_assert_valid_procids< Node >(), libMesh::MeshTools::libmesh_assert_valid_refinement_flags(), libMesh::DofMap::local_variable_indices(), libMesh::MeshBase::n_active_elem_on_proc(), libMesh::MeshBase::n_elem_on_proc(), libMesh::MeshBase::n_nodes_on_proc(), libMesh::Partitioner::partition(), libMesh::MeshBase::partition(), libMesh::Partitioner::partition_unpartitioned_elements(), libMesh::PetscLinearSolver< T >::PetscLinearSolver(), libMesh::System::point_gradient(), libMesh::System::point_hessian(), libMesh::System::point_value(), libMesh::MeshTools::processor_bounding_box(), libMesh::System::project_vector(), libMesh::NameBasedIO::read(), libMesh::Nemesis_IO::read(), libMesh::CheckpointIO::read(), libMesh::XdrIO::read(), libMesh::System::read_parallel_data(), libMesh::System::read_SCALAR_dofs(), libMesh::System::read_serialized_blocked_dof_objects(), libMesh::System::read_serialized_vector(), libMesh::Partitioner::repartition(), libMesh::Partitioner::set_node_processor_ids(), libMesh::DofMap::set_nonlocal_dof_objects(), libMesh::BoundaryInfo::sync(), libMesh::ParallelMesh::update_parallel_id_counts(), libMesh::CheckpointIO::write(), libMesh::GMVIO::write_binary(), libMesh::GMVIO::write_discontinuous_gmv(), libMesh::System::write_parallel_data(), libMesh::System::write_SCALAR_dofs(), libMesh::XdrIO::write_serialized_bcs(), libMesh::System::write_serialized_blocked_dof_objects(), libMesh::XdrIO::write_serialized_connectivity(), libMesh::XdrIO::write_serialized_nodes(), and libMesh::XdrIO::write_serialized_nodesets().

  { return cast_int<processor_id_type>(_communicator.size()); }
void libMesh::SparsityPattern::Build::operator() ( const ConstElemRange range)

Definition at line 2457 of file dof_map.C.

References libMesh::Elem::active_family_tree_by_neighbor(), libMesh::StoredRange< iterator_type, object_type >::begin(), dof_coupling, libMesh::DofMap::dof_indices(), dof_map, libMesh::CouplingMatrix::empty(), libMesh::StoredRange< iterator_type, object_type >::end(), libMesh::DofMap::end_dof(), libMesh::DofMap::find_connected_dofs(), libMesh::DofMap::first_dof(), implicit_neighbor_dofs, libMesh::libmesh_assert(), libMesh::DofMap::n_dofs_on_processor(), n_nz, n_oz, libMesh::Elem::n_sides(), libMesh::DofMap::n_variables(), need_full_sparsity_pattern, libMesh::Elem::neighbor(), nonlocal_pattern, libMesh::ParallelObject::processor_id(), libMesh::CouplingMatrix::size(), libMesh::SparsityPattern::sort_row(), and sparsity_pattern.

{
  // Compute the sparsity structure of the global matrix.  This can be
  // fed into a PetscMatrix to allocate exacly the number of nonzeros
  // necessary to store the matrix.  This algorithm should be linear
  // in the (# of elements)*(# nodes per element)
  const processor_id_type proc_id           = mesh.processor_id();
  const dof_id_type n_dofs_on_proc    = dof_map.n_dofs_on_processor(proc_id);
  const dof_id_type first_dof_on_proc = dof_map.first_dof(proc_id);
  const dof_id_type end_dof_on_proc   = dof_map.end_dof(proc_id);

  sparsity_pattern.resize(n_dofs_on_proc);

  // If the user did not explicitly specify the DOF coupling
  // then all the DOFS are coupled to each other.  Furthermore,
  // we can take a shortcut and do this more quickly here.  So
  // we use an if-test.
  if ((dof_coupling == NULL) || (dof_coupling->empty()))
    {
      std::vector<dof_id_type>
        element_dofs,
        neighbor_dofs,
        dofs_to_add;

      std::vector<const Elem*> active_neighbors;

      for (ConstElemRange::const_iterator elem_it = range.begin() ; elem_it != range.end(); ++elem_it)
        {
          const Elem* const elem = *elem_it;

          // Get the global indices of the DOFs with support on this element
          dof_map.dof_indices (elem, element_dofs);
#ifdef LIBMESH_ENABLE_CONSTRAINTS
          dof_map.find_connected_dofs (element_dofs);
#endif

          // We can be more efficient if we sort the element DOFs
          // into increasing order
          std::sort(element_dofs.begin(), element_dofs.end());

          const unsigned int n_dofs_on_element =
            cast_int<unsigned int>(element_dofs.size());

          for (unsigned int i=0; i<n_dofs_on_element; i++)
            {
              const dof_id_type ig = element_dofs[i];

              SparsityPattern::Row *row;

              // We save non-local row components for now so we can
              // communicate them to other processors later.

              if ((ig >= first_dof_on_proc) &&
                  (ig <  end_dof_on_proc))
                {
                  // This is what I mean
                  // libmesh_assert_greater_equal ((ig - first_dof_on_proc), 0);
                  // but do the test like this because ig and
                  // first_dof_on_proc are unsigned ints
                  libmesh_assert_greater_equal (ig, first_dof_on_proc);
                  libmesh_assert_less ((ig - first_dof_on_proc), sparsity_pattern.size());

                  row = &sparsity_pattern[ig - first_dof_on_proc];
                }
              else
                {
                  row = &nonlocal_pattern[ig];
                }

              // If the row is empty we will add *all* the element DOFs,
              // so just do that.
              if (row->empty())
                {
                  row->insert(row->end(),
                              element_dofs.begin(),
                              element_dofs.end());
                }
              else
                {
                  // Build a list of the DOF indices not found in the
                  // sparsity pattern
                  dofs_to_add.clear();

                  // Cache iterators.  Low will move forward, subsequent
                  // searches will be on smaller ranges
                  SparsityPattern::Row::iterator
                    low  = std::lower_bound (row->begin(), row->end(), element_dofs.front()),
                    high = std::upper_bound (low,          row->end(), element_dofs.back());

                  for (unsigned int j=0; j<n_dofs_on_element; j++)
                    {
                      const dof_id_type jg = element_dofs[j];

                      // See if jg is in the sorted range
                      std::pair<SparsityPattern::Row::iterator,
                        SparsityPattern::Row::iterator>
                        pos = std::equal_range (low, high, jg);

                      // Must add jg if it wasn't found
                      if (pos.first == pos.second)
                        dofs_to_add.push_back(jg);

                      // pos.first is now a valid lower bound for any
                      // remaining element DOFs. (That's why we sorted them.)
                      // Use it for the next search
                      low = pos.first;
                    }

                  // Add to the sparsity pattern
                  if (!dofs_to_add.empty())
                    {
                      const std::size_t old_size = row->size();

                      row->insert (row->end(),
                                   dofs_to_add.begin(),
                                   dofs_to_add.end());

                      SparsityPattern::sort_row
                        (row->begin(), row->begin()+old_size, row->end());
                    }
                }

              // Now (possibly) add dofs from neighboring elements
              // TODO:[BSK] optimize this like above!
              if (implicit_neighbor_dofs)
                for (unsigned int s=0; s<elem->n_sides(); s++)
                  if (elem->neighbor(s) != NULL)
                    {
                      const Elem* const neighbor_0 = elem->neighbor(s);
#ifdef LIBMESH_ENABLE_AMR
                      neighbor_0->active_family_tree_by_neighbor(active_neighbors,elem);
#else
                      active_neighbors.clear();
                      active_neighbors.push_back(neighbor_0);
#endif

                      for (std::size_t a=0; a != active_neighbors.size(); ++a)
                        {
                          const Elem *neighbor = active_neighbors[a];

                          dof_map.dof_indices (neighbor, neighbor_dofs);
#ifdef LIBMESH_ENABLE_CONSTRAINTS
                          dof_map.find_connected_dofs (neighbor_dofs);
#endif
                          const std::size_t n_dofs_on_neighbor = neighbor_dofs.size();

                          for (std::size_t j=0; j<n_dofs_on_neighbor; j++)
                            {
                              const dof_id_type jg = neighbor_dofs[j];

                              // See if jg is in the sorted range
                              std::pair<SparsityPattern::Row::iterator,
                                SparsityPattern::Row::iterator>
                                pos = std::equal_range (row->begin(), row->end(), jg);

                              // Insert jg if it wasn't found
                              if (pos.first == pos.second)
                                row->insert (pos.first, jg);
                            }
                        }
                    }
            }
        }
    }

  // This is what we do in the case that the user has specified
  // explicit DOF coupling.
  else
    {
      libmesh_assert(dof_coupling);
      libmesh_assert_equal_to (dof_coupling->size(),
                               dof_map.n_variables());

      const unsigned int n_var = dof_map.n_variables();

      std::vector<dof_id_type>
        element_dofs_i,
        element_dofs_j,
        neighbor_dofs,
        dofs_to_add;


      std::vector<const Elem*> active_neighbors;
      for (ConstElemRange::const_iterator elem_it = range.begin() ; elem_it != range.end(); ++elem_it)
        for (unsigned int vi=0; vi<n_var; vi++)
          {
            const Elem* const elem = *elem_it;

            // Find element dofs for variable vi
            dof_map.dof_indices (elem, element_dofs_i, vi);
#ifdef LIBMESH_ENABLE_CONSTRAINTS
            dof_map.find_connected_dofs (element_dofs_i);
#endif

            // We can be more efficient if we sort the element DOFs
            // into increasing order
            std::sort(element_dofs_i.begin(), element_dofs_i.end());
            const unsigned int n_dofs_on_element_i =
              cast_int<unsigned int>(element_dofs_i.size());

            for (unsigned int vj=0; vj<n_var; vj++)
              if ((*dof_coupling)(vi,vj)) // If vi couples to vj
                {
                  // Find element dofs for variable vj, note that
                  // if vi==vj we already have the dofs.
                  if (vi != vj)
                    {
                      dof_map.dof_indices (elem, element_dofs_j, vj);
#ifdef LIBMESH_ENABLE_CONSTRAINTS
                      dof_map.find_connected_dofs (element_dofs_j);
#endif

                      // We can be more efficient if we sort the element DOFs
                      // into increasing order
                      std::sort (element_dofs_j.begin(), element_dofs_j.end());
                    }
                  else
                    element_dofs_j = element_dofs_i;

                  const unsigned int n_dofs_on_element_j =
                    cast_int<unsigned int>(element_dofs_j.size());

                  // there might be 0 dofs for the other variable on the same element (when subdomain variables do not overlap) and that's when we do not do anything
                  if (n_dofs_on_element_j > 0)
                    {
                      for (unsigned int i=0; i<n_dofs_on_element_i; i++)
                        {
                          const dof_id_type ig = element_dofs_i[i];

                          SparsityPattern::Row *row;

                          // We save non-local row components for now so we can
                          // communicate them to other processors later.

                          if ((ig >= first_dof_on_proc) &&
                              (ig <  end_dof_on_proc))
                            {
                              // This is what I mean
                              // libmesh_assert_greater_equal ((ig - first_dof_on_proc), 0);
                              // but do the test like this because ig and
                              // first_dof_on_proc are unsigned ints
                              libmesh_assert_greater_equal (ig, first_dof_on_proc);
                              libmesh_assert_less (ig, (sparsity_pattern.size() +
                                                        first_dof_on_proc));

                              row = &sparsity_pattern[ig - first_dof_on_proc];
                            }
                          else
                            {
                              row = &nonlocal_pattern[ig];
                            }

                          // If the row is empty we will add *all* the element j DOFs,
                          // so just do that.
                          if (row->empty())
                            {
                              row->insert(row->end(),
                                          element_dofs_j.begin(),
                                          element_dofs_j.end());
                            }
                          else
                            {
                              // Build a list of the DOF indices not found in the
                              // sparsity pattern
                              dofs_to_add.clear();

                              // Cache iterators.  Low will move forward, subsequent
                              // searches will be on smaller ranges
                              SparsityPattern::Row::iterator
                                low  = std::lower_bound
                                (row->begin(), row->end(), element_dofs_j.front()),
                                high = std::upper_bound
                                (low,          row->end(), element_dofs_j.back());

                              for (unsigned int j=0; j<n_dofs_on_element_j; j++)
                                {
                                  const dof_id_type jg = element_dofs_j[j];

                                  // See if jg is in the sorted range
                                  std::pair<SparsityPattern::Row::iterator,
                                    SparsityPattern::Row::iterator>
                                    pos = std::equal_range (low, high, jg);

                                  // Must add jg if it wasn't found
                                  if (pos.first == pos.second)
                                    dofs_to_add.push_back(jg);

                                  // pos.first is now a valid lower bound for any
                                  // remaining element j DOFs. (That's why we sorted them.)
                                  // Use it for the next search
                                  low = pos.first;
                                }

                              // Add to the sparsity pattern
                              if (!dofs_to_add.empty())
                                {
                                  const std::size_t old_size = row->size();

                                  row->insert (row->end(),
                                               dofs_to_add.begin(),
                                               dofs_to_add.end());

                                  SparsityPattern::sort_row
                                    (row->begin(), row->begin()+old_size,
                                     row->end());
                                }
                            }
                          // Now (possibly) add dofs from neighboring elements
                          // TODO:[BSK] optimize this like above!
                          if (implicit_neighbor_dofs)
                            for (unsigned int s=0; s<elem->n_sides(); s++)
                              if (elem->neighbor(s) != NULL)
                                {
                                  const Elem* const neighbor_0 = elem->neighbor(s);
#ifdef LIBMESH_ENABLE_AMR
                                  neighbor_0->active_family_tree_by_neighbor(active_neighbors,elem);
#else
                                  active_neighbors.clear();
                                  active_neighbors.push_back(neighbor_0);
#endif

                                  for (std::size_t a=0; a != active_neighbors.size(); ++a)
                                    {
                                      const Elem *neighbor = active_neighbors[a];

                                      dof_map.dof_indices (neighbor, neighbor_dofs);
#ifdef LIBMESH_ENABLE_CONSTRAINTS
                                      dof_map.find_connected_dofs (neighbor_dofs);
#endif
                                      const unsigned int n_dofs_on_neighbor =
                                        cast_int<unsigned int>(neighbor_dofs.size());

                                      for (unsigned int j=0; j<n_dofs_on_neighbor; j++)
                                        {
                                          const dof_id_type jg = neighbor_dofs[j];

                                          // See if jg is in the sorted range
                                          std::pair<SparsityPattern::Row::iterator,
                                            SparsityPattern::Row::iterator>
                                            pos = std::equal_range (row->begin(), row->end(), jg);

                                          // Insert jg if it wasn't found
                                          if (pos.first == pos.second)
                                            row->insert (pos.first, jg);
                                        }
                                    }
                                }
                        }
                    }
                }
          }
    }

  // Now a new chunk of sparsity structure is built for all of the
  // DOFs connected to our rows of the matrix.

  // If we're building a full sparsity pattern, then we've got
  // complete rows to work with, so we can just count them from
  // scratch.
  if (need_full_sparsity_pattern)
    {
      n_nz.clear();
      n_oz.clear();
    }

  n_nz.resize (n_dofs_on_proc, 0);
  n_oz.resize (n_dofs_on_proc, 0);

  for (dof_id_type i=0; i<n_dofs_on_proc; i++)
    {
      // Get the row of the sparsity pattern
      SparsityPattern::Row &row = sparsity_pattern[i];

      for (dof_id_type j=0; j<row.size(); j++)
        if ((row[j] < first_dof_on_proc) || (row[j] >= end_dof_on_proc))
          n_oz[i]++;
        else
          n_nz[i]++;

      // If we're not building a full sparsity pattern, then we want
      // to avoid overcounting these entries as much as possible.
      if (!need_full_sparsity_pattern)
        row.clear();
    }
}

Definition at line 2948 of file dof_map.C.

References dof_id, libMesh::libmesh_assert(), std::min(), libMesh::n_processors(), and libMesh::processor_id().

{
  parallel_object_only();
  this->comm().verify(need_full_sparsity_pattern);

  const dof_id_type n_global_dofs   = dof_map.n_dofs();
  const dof_id_type n_dofs_on_proc  = dof_map.n_dofs_on_processor(this->processor_id());
  const dof_id_type local_first_dof = dof_map.first_dof();
  const dof_id_type local_end_dof   = dof_map.end_dof();

  // Trade sparsity rows with other processors
  for (processor_id_type p=1; p != this->n_processors(); ++p)
    {
      // Push to processor procup while receiving from procdown
      processor_id_type procup =
        cast_int<processor_id_type>((this->processor_id() + p) %
                                    this->n_processors());
      processor_id_type procdown =
        cast_int<processor_id_type>((this->n_processors() + this->processor_id() - p) %
                                    this->n_processors());

      // Pack the sparsity pattern rows to push to procup
      std::vector<dof_id_type> pushed_row_ids,
        pushed_row_ids_to_me;
      std::vector<std::vector<dof_id_type> > pushed_rows,
        pushed_rows_to_me;

      // Move nonlocal row information to a structure to send it from;
      // we don't need it in the map after that.
      NonlocalGraph::iterator it = nonlocal_pattern.begin();
      while (it != nonlocal_pattern.end())
        {
          const dof_id_type dof_id = it->first;
          processor_id_type proc_id = 0;
          while (dof_id >= dof_map.end_dof(proc_id))
            proc_id++;

          libmesh_assert (proc_id != this->processor_id());

          if (proc_id == procup)
            {
              pushed_row_ids.push_back(dof_id);

              // We can't just do the swap trick here, thanks to the
              // differing vector allocators?
              pushed_rows.push_back(std::vector<dof_id_type>());
              pushed_rows.back().assign
                (it->second.begin(), it->second.end());

              nonlocal_pattern.erase(it++);
            }
          else
            ++it;
        }

      this->comm().send_receive(procup, pushed_row_ids,
                                procdown, pushed_row_ids_to_me);
      this->comm().send_receive(procup, pushed_rows,
                                procdown, pushed_rows_to_me);
      pushed_row_ids.clear();
      pushed_rows.clear();

      const std::size_t n_rows = pushed_row_ids_to_me.size();
      for (std::size_t i=0; i != n_rows; ++i)
        {
          const dof_id_type r = pushed_row_ids_to_me[i];
          const dof_id_type my_r = r - local_first_dof;

          std::vector<dof_id_type> &their_row = pushed_rows_to_me[i];

          if (need_full_sparsity_pattern)
            {
              SparsityPattern::Row &my_row =
                sparsity_pattern[my_r];

              // They wouldn't have sent an empty row
              libmesh_assert(!their_row.empty());

              // We can end up with an empty row on a dof that touches our
              // inactive elements but not our active ones
              if (my_row.empty())
                {
                  my_row.assign (their_row.begin(),
                                 their_row.end());
                }
              else
                {
                  my_row.insert (my_row.end(),
                                 their_row.begin(),
                                 their_row.end());

                  // We cannot use SparsityPattern::sort_row() here because it expects
                  // the [begin,middle) [middle,end) to be non-overlapping.  This is not
                  // necessarily the case here, so use std::sort()
                  std::sort (my_row.begin(), my_row.end());

                  my_row.erase(std::unique (my_row.begin(), my_row.end()), my_row.end());
                }

              // fix the number of on and off-processor nonzeros in this row
              n_nz[my_r] = n_oz[my_r] = 0;

              for (std::size_t j=0; j<my_row.size(); j++)
                if ((my_row[j] < local_first_dof) || (my_row[j] >= local_end_dof))
                  n_oz[my_r]++;
                else
                  n_nz[my_r]++;
            }
          else
            {
              for (std::size_t j=0; j<their_row.size(); j++)
                if ((their_row[j] < local_first_dof) || (their_row[j] >= local_end_dof))
                  n_oz[my_r]++;
                else
                  n_nz[my_r]++;

              n_nz[my_r] = std::min(n_nz[my_r], n_dofs_on_proc);
              n_oz[my_r] = std::min(n_oz[my_r],
                                    static_cast<dof_id_type>(n_global_dofs-n_nz[my_r]));
            }
        }
    }

  // We should have sent everything at this point.
  libmesh_assert (nonlocal_pattern.empty());
}
Returns:
the rank of this processor in the group.

Definition at line 98 of file parallel_object.h.

References libMesh::ParallelObject::_communicator, and libMesh::Parallel::Communicator::rank().

Referenced by libMesh::MetisPartitioner::_do_partition(), libMesh::EquationSystems::_read_impl(), libMesh::SerialMesh::active_local_elements_begin(), libMesh::ParallelMesh::active_local_elements_begin(), libMesh::SerialMesh::active_local_elements_end(), libMesh::ParallelMesh::active_local_elements_end(), libMesh::SerialMesh::active_local_subdomain_elements_begin(), libMesh::ParallelMesh::active_local_subdomain_elements_begin(), libMesh::SerialMesh::active_local_subdomain_elements_end(), libMesh::ParallelMesh::active_local_subdomain_elements_end(), libMesh::SerialMesh::active_not_local_elements_begin(), libMesh::ParallelMesh::active_not_local_elements_begin(), libMesh::SerialMesh::active_not_local_elements_end(), libMesh::ParallelMesh::active_not_local_elements_end(), libMesh::ParallelMesh::add_elem(), libMesh::DofMap::add_neighbors_to_send_list(), libMesh::ParallelMesh::add_node(), libMesh::UnstructuredMesh::all_second_order(), libMesh::FEMSystem::assembly(), libMesh::ParmetisPartitioner::assign_partitioning(), libMesh::ParallelMesh::assign_unique_ids(), libMesh::EquationSystems::build_discontinuous_solution_vector(), libMesh::Nemesis_IO_Helper::build_element_and_node_maps(), libMesh::ParmetisPartitioner::build_graph(), libMesh::InfElemBuilder::build_inf_elem(), libMesh::DofMap::build_sparsity(), libMesh::ParallelMesh::clear(), libMesh::ExodusII_IO_Helper::close(), libMesh::Nemesis_IO_Helper::compute_border_node_ids(), libMesh::Nemesis_IO_Helper::compute_communication_map_parameters(), libMesh::Nemesis_IO_Helper::compute_internal_and_border_elems_and_internal_nodes(), libMesh::Nemesis_IO_Helper::compute_node_communication_maps(), libMesh::Nemesis_IO_Helper::compute_num_global_elem_blocks(), libMesh::Nemesis_IO_Helper::compute_num_global_nodesets(), libMesh::Nemesis_IO_Helper::compute_num_global_sidesets(), libMesh::Nemesis_IO_Helper::construct_nemesis_filename(), libMesh::ExodusII_IO_Helper::create(), libMesh::ParallelMesh::delete_elem(), libMesh::ParallelMesh::delete_node(), libMesh::DofMap::distribute_dofs(), libMesh::DofMap::distribute_local_dofs_node_major(), libMesh::DofMap::distribute_local_dofs_var_major(), libMesh::DofMap::end_dof(), libMesh::DofMap::end_old_dof(), libMesh::EnsightIO::EnsightIO(), libMesh::SerialMesh::facelocal_elements_begin(), libMesh::ParallelMesh::facelocal_elements_begin(), libMesh::SerialMesh::facelocal_elements_end(), libMesh::ParallelMesh::facelocal_elements_end(), libMesh::MeshFunction::find_element(), libMesh::UnstructuredMesh::find_neighbors(), libMesh::DofMap::first_dof(), libMesh::DofMap::first_old_dof(), libMesh::Nemesis_IO_Helper::get_cmap_params(), libMesh::Nemesis_IO_Helper::get_eb_info_global(), libMesh::Nemesis_IO_Helper::get_elem_cmap(), libMesh::Nemesis_IO_Helper::get_elem_map(), libMesh::MeshBase::get_info(), libMesh::Nemesis_IO_Helper::get_init_global(), libMesh::Nemesis_IO_Helper::get_init_info(), libMesh::Nemesis_IO_Helper::get_loadbal_param(), libMesh::Nemesis_IO_Helper::get_node_cmap(), libMesh::Nemesis_IO_Helper::get_node_map(), libMesh::Nemesis_IO_Helper::get_ns_param_global(), libMesh::Nemesis_IO_Helper::get_ss_param_global(), libMesh::SystemSubsetBySubdomain::init(), libMesh::ParmetisPartitioner::initialize(), libMesh::ExodusII_IO_Helper::initialize(), libMesh::ExodusII_IO_Helper::initialize_element_variables(), libMesh::ExodusII_IO_Helper::initialize_global_variables(), libMesh::ExodusII_IO_Helper::initialize_nodal_variables(), libMesh::ParallelMesh::insert_elem(), join(), libMesh::DofMap::last_dof(), libMesh::MeshTools::libmesh_assert_valid_procids< Elem >(), libMesh::MeshTools::libmesh_assert_valid_procids< Node >(), libMesh::SerialMesh::local_elements_begin(), libMesh::ParallelMesh::local_elements_begin(), libMesh::SerialMesh::local_elements_end(), libMesh::ParallelMesh::local_elements_end(), libMesh::SerialMesh::local_level_elements_begin(), libMesh::ParallelMesh::local_level_elements_begin(), libMesh::SerialMesh::local_level_elements_end(), libMesh::ParallelMesh::local_level_elements_end(), libMesh::SerialMesh::local_nodes_begin(), libMesh::ParallelMesh::local_nodes_begin(), libMesh::SerialMesh::local_nodes_end(), libMesh::ParallelMesh::local_nodes_end(), libMesh::SerialMesh::local_not_level_elements_begin(), libMesh::ParallelMesh::local_not_level_elements_begin(), libMesh::SerialMesh::local_not_level_elements_end(), libMesh::ParallelMesh::local_not_level_elements_end(), libMesh::DofMap::local_variable_indices(), libMesh::MeshRefinement::make_coarsening_compatible(), libMesh::MeshBase::n_active_local_elem(), libMesh::BoundaryInfo::n_boundary_conds(), libMesh::BoundaryInfo::n_edge_conds(), libMesh::DofMap::n_local_dofs(), libMesh::System::n_local_dofs(), libMesh::MeshBase::n_local_elem(), libMesh::MeshBase::n_local_nodes(), libMesh::BoundaryInfo::n_nodeset_conds(), libMesh::SerialMesh::not_local_elements_begin(), libMesh::ParallelMesh::not_local_elements_begin(), libMesh::SerialMesh::not_local_elements_end(), libMesh::ParallelMesh::not_local_elements_end(), libMesh::WeightedPatchRecoveryErrorEstimator::EstimateError::operator()(), operator()(), libMesh::PatchRecoveryErrorEstimator::EstimateError::operator()(), libMesh::ParallelMesh::ParallelMesh(), libMesh::System::point_gradient(), libMesh::System::point_hessian(), libMesh::System::point_value(), libMesh::System::project_vector(), libMesh::Nemesis_IO_Helper::put_cmap_params(), libMesh::Nemesis_IO_Helper::put_elem_cmap(), libMesh::Nemesis_IO_Helper::put_elem_map(), libMesh::Nemesis_IO_Helper::put_loadbal_param(), libMesh::Nemesis_IO_Helper::put_node_cmap(), libMesh::Nemesis_IO_Helper::put_node_map(), libMesh::NameBasedIO::read(), libMesh::Nemesis_IO::read(), libMesh::CheckpointIO::read(), libMesh::XdrIO::read(), libMesh::ExodusII_IO_Helper::read_elem_num_map(), libMesh::System::read_header(), libMesh::System::read_legacy_data(), libMesh::ExodusII_IO_Helper::read_node_num_map(), libMesh::System::read_parallel_data(), libMesh::System::read_SCALAR_dofs(), libMesh::XdrIO::read_serialized_bc_names(), libMesh::XdrIO::read_serialized_bcs(), libMesh::System::read_serialized_blocked_dof_objects(), libMesh::XdrIO::read_serialized_connectivity(), libMesh::System::read_serialized_data(), libMesh::XdrIO::read_serialized_nodes(), libMesh::XdrIO::read_serialized_nodesets(), libMesh::XdrIO::read_serialized_subdomain_names(), libMesh::System::read_serialized_vector(), libMesh::System::read_serialized_vectors(), libMesh::MeshData::read_xdr(), libMesh::SerialMesh::semilocal_elements_begin(), libMesh::ParallelMesh::semilocal_elements_begin(), libMesh::SerialMesh::semilocal_elements_end(), libMesh::ParallelMesh::semilocal_elements_end(), libMesh::Partitioner::set_node_processor_ids(), libMesh::DofMap::set_nonlocal_dof_objects(), libMesh::LaplaceMeshSmoother::smooth(), libMesh::BoundaryInfo::sync(), libMesh::MeshTools::total_weight(), libMesh::ParallelMesh::update_parallel_id_counts(), libMesh::MeshTools::weight(), libMesh::NameBasedIO::write(), libMesh::ExodusII_IO::write(), libMesh::CheckpointIO::write(), libMesh::XdrIO::write(), libMesh::EquationSystems::write(), libMesh::GMVIO::write_discontinuous_gmv(), libMesh::ExodusII_IO::write_element_data(), libMesh::ExodusII_IO_Helper::write_element_values(), libMesh::ExodusII_IO_Helper::write_elements(), libMesh::ExodusII_IO::write_global_data(), libMesh::ExodusII_IO_Helper::write_global_values(), libMesh::System::write_header(), libMesh::ExodusII_IO::write_information_records(), libMesh::ExodusII_IO_Helper::write_information_records(), libMesh::ExodusII_IO_Helper::write_nodal_coordinates(), libMesh::UCDIO::write_nodal_data(), libMesh::ExodusII_IO::write_nodal_data(), libMesh::ExodusII_IO::write_nodal_data_discontinuous(), libMesh::ExodusII_IO_Helper::write_nodal_values(), libMesh::ExodusII_IO_Helper::write_nodesets(), libMesh::Nemesis_IO_Helper::write_nodesets(), libMesh::System::write_parallel_data(), libMesh::System::write_SCALAR_dofs(), libMesh::XdrIO::write_serialized_bc_names(), libMesh::XdrIO::write_serialized_bcs(), libMesh::System::write_serialized_blocked_dof_objects(), libMesh::XdrIO::write_serialized_connectivity(), libMesh::System::write_serialized_data(), libMesh::XdrIO::write_serialized_nodes(), libMesh::XdrIO::write_serialized_nodesets(), libMesh::XdrIO::write_serialized_subdomain_names(), libMesh::System::write_serialized_vector(), libMesh::System::write_serialized_vectors(), libMesh::ExodusII_IO_Helper::write_sidesets(), libMesh::Nemesis_IO_Helper::write_sidesets(), libMesh::ExodusII_IO::write_timestep(), and libMesh::ExodusII_IO_Helper::write_timestep().

  { return cast_int<processor_id_type>(_communicator.rank()); }

Member Data Documentation

Definition at line 84 of file sparsity_pattern.h.

Referenced by operator()().

Definition at line 83 of file sparsity_pattern.h.

Referenced by operator()().

Definition at line 85 of file sparsity_pattern.h.

Referenced by operator()().

Definition at line 82 of file sparsity_pattern.h.

Definition at line 93 of file sparsity_pattern.h.

Referenced by join(), and operator()().

Definition at line 94 of file sparsity_pattern.h.

Referenced by join(), and operator()().

Definition at line 86 of file sparsity_pattern.h.

Referenced by operator()().


The documentation for this class was generated from the following files: