A central challenging problem in humanoid robotics is to plan and execute dynamic tasks in changing environments, and at the same time keep the result convincing and realistic. Sampling-based online motion planners are particularly powerful for automatically generating collision-free motions in changing environments. However, without learning strategies, each task still has to be planned from scratch, preventing these algorithms from getting closer to realtime performance. Moreover, the nature of the random sampling strategy employed in these planners also results in extremely non human-like solutions. This document addresses these two issues by proposing to learn important features from previously planned solutions, or from real captured motion to improve both the efficiency and the solution quality. Our methods work in changing environments, where obstacles can have different positions in different tasks. However, we assume that obstacles are static during the execution of a single task. We first propose the Attractor Guided Planner (AGP), which extends existing motion planners in two simple but important ways. First, it extracts significant attractor points from successful paths as guiding landmarks for new similar tasks. Second, it relies on a task comparison metric to decide when previous solutions should be reused to guide the planning of new tasks. The task comparison metric takes into account the task specification and as well environment features which are relevant to the query. With combination of motion capture technique, the AGP planner also shows big improvements towards generating realistic planned motions. We propose a constraint detection method that applies to humanoid manipulation tasks. After recording a performer's demonstrated motion, our method will automatically detect important constraints, and then segment the input motion according to different types of constraints. Attractors are placed at the connections between each pair of segments and assigned the same constraints as the previous segment. Then, given a new similar task, the new planning is guided not only toward the locations of the attractors, but also preserving the constraints of the attractors. Several experiments are presented with different humanoid reaching examples where obstacles are differently located for each task. Our results show that the AGP greatly improves both the planning time and solution quality, when comparing to traditional sampling-based motion planners. We also show that with our constraint detection method, the AGP planner can efficiently find a solution that preserves the features of the input motion, making the solution motion coherent with the task being solved and therefore more realistic. Although our current results are not yet capable of achieving real-time performance nor overall realistic humanlike motions, we believe that the techniques introduced here are key for getting closer to these goals.